
It’s all about the ecosystem – build and nurture yours

A friend recently pointed me to the book They Will Be Giants by Robert Wilson and I immediately saw why. Most of my friends know that my ideal job is working in a business with any of them. Regardless of role or financial success, just working with them has always been inspirational. I’ve always stated […]
Learnings from TigerGraph and Expero webinar

As mentioned Monday, I have a week of different webinars lined up to watch and learn. If you missed the first two, I’m sure you can find it on-demand in the original links. Here is the first one I watched – from TigerGraph and Expero – with their key talking points for context and my takeaways added. #1 – Energy […]
4 Webinars This Week – GPU, 5G, graph analytics, cloud

4 Webinars I’m Watching This Week – GPU, 5G, graph analytics, cloud… Accelerating Data Science Workflows With Rapids How RAPIDS accelerates your Python data science toolchain with minimal code changes and no new tools to learn. How RAPIDS can accelerate model training and time to deployment. The wealth of accelerated apps available to maximize datacenter […]
Zeppelin Notebook Quick Start on OSX v0.5.6

This is a follow-up to my post from last year Apache Zeppelin on OSX – Ultra Quick Start but without building from source. Today I tested the latest version of Zeppelin (0.5.6) and, using their distributed binaries, was instantly able to launch Zeppelin and run both Scala and Python jobs on my Macbook. This was with zero configuration, […]
Spark Analysis of Global Place Names (GeoNames)

Spark Analysis on a Large File GeoNames.org has free gazetteer data by country or for the world, provided in tab-separated text files. In this post I show you how to do some simple analysis using DataFrames in Spark. As the global file is 280M compressed and 1.2G uncompressed. This size of file makes it difficult to […]
Serverspec checks settings on a Hadoop cluster

Serverspec is a Ruby-based system that can run Rspec formatted tests against a host. It can check for a long list of system details and status such as total memory, CPU count, running services, and more, including custom shell callouts. It compares the results of the checks with predefined thresholds and reports a pass or […]
Hadoop Options for SQL Databases

Drowning while trying to understand your options for SQL-based database management in Hadoop? This graphic is a simplified comparison of the various features of several popular products being used today. I outline some of my biggest differentiators in this post. While this is a marketing slide for Actian’s SQL in Hadoop enterprise solution, I wish I saw it earlier so I could […]
“Big Data” off 2015 Hype Cycle?

See this official 2015 hype cycle video here to get it straight from Gartner. In the video she says, first, it’s passed over the hump and is no longer just hype. Second, it’s embedded within other items throughout the cycle now. I can understand how this can get confusing to track and qualify, but isn’t […]
Spatial Data Made Useful

Everyone that deals with geographic/spatial/geospatial data knows they need specialised tools for the job. Fortunately, there are a ton of open source tools up for the challenge. In my latest book, Geospatial Power Tools, I show how to use an advanced set of command line tools that you can start using today. From re-projecting point coordinates […]
Common Zeppelin Errors

A few different errors have popped during my initiation into Apache Zeppelin, here are a few of them, summarised with workarounds if you need them. Tutorial Failure Due To Spark Versions Default Zeppelin comes with Spark 1.1 (though it may be updated by the time you read this). The current Zeppelin tutorial assumes Spark 1.3 or greater […]
Python Spark SQL – Zeppelin Tutorial – No Scala

My latest notebook aims to mimic the original Scala-based Spark SQL tutorial with one that uses Python instead. Above you can see the two parallel translations side-by-side. Python Spark SQL Tutorial Code Here is the resulting Python data loading code. The SQL code is identical to the Tutorial notebook, so copy and paste if you need it. I […]
Zeppelin Notebook Tutorial Walkthrough

This is my short video (14 min) showing how to build and launch the Apache Zeppelin notebook platform – a web UI for interactive query and analysis. This is all done running locally via OSX on a Macbook. In this video we focus on using the tutorial notebook that comes with Zeppelin and discuss each step – including interactive querying and charting – […]
Apache Zeppelin on OSX – Ultra Quick Start

The Zeppelin project provides a powerful web-based notebook platform for data analysis and discovery. Behind the scenes it supports Spark distributed contexts as well as other language bindings on top of Spark. This post is a very simple introduction to show the first few steps to get started. You’ll find all you need to know […]
Partitioned Data & Why It Matters

As data volumes grow, so does your need to understand how to partition your data. Until you understand this distributed storage concept, you will be unable to choose the best approach for the job. This post gives an introductory explanation of partitioning and you will see why it is integral to the Hadoop Distributed File System (HDFS) increasingly […]
Home Energy Monitor Series – Internet of Things

Recently I started an Internet of Things series on my experiences installing, using and analysing data from a smart electrical meter. This included a BC Hydro smart meter, Eagle monitoring gateway from rainforest automation, and a cloud-based analytics service from Bidgely. I’ve collated all the posts on the topic for you below. More will be […]