Big Data in Boston

Last week, I had the opportunity to attend the Xconomy Big Data forum in Boston.  It was absolutely worthwhile and I wanted to share some perspective on Big Data, with an enterprise delivery focus.

In the spirit of all the questions for speakers and moderated panels, this post is organized as Questions and Responses.

1.  What’s the definition of Big Data?

R:  Many took a light-hearted approach to defining the term, highlighting some of the key descriptive words – volume, velocity, variety, analytics, and petabytes.  There wasn't a consensus definition but a recurring point was that, after looking at what’s out there, a good definition probably doesn't exist.  Some even wondered if the term itself was problematic.  Others suggested that the definitions are, in part, the result of marketing and sales efforts.

What became clear was that the specific definition was not all that important.  What’s needed is knowledge of key aspects of “Big Data” in order to answer additional questions.

2.  What’s currently happening with Big Data?

R:  A bunch.  From start-up firms interested in specific challenges like Bitly, Hopper, and Objective Logistics to big tech firms investing and expanding capabilities like HP, EMC and others.  There are impacts and implications related to infrastructure (aggregation, storage, and delivery) and data use and insights for consumers, companies, and even the government (via software).

As a program and project delivery professional everything I heard further validated a sense in me that Big Data, however it's defined, is real, very real.

http://www.xconomy.com/boston/2012/10/29/future-of-big-data-in-boston-the-photos/ 

3.  What does the future hold for Big Data?

R:  Here’s where people’s passions clearly shown through.  Many of the speakers and panelists were up to their necks investing in solutions or driven to solve the challenges associated with Big Data.  Of course, there was near consensus that big advancements were on the horizon, after all this was a Big Data event.  But interestingly, there was a split on whether or not this would unfold like the revolution that has been mobile the past few years, or be more of a slow burn due to barriers like the difficult emerging data-science and the lack of enough talented “super” engineers needed to address the work ahead.

Personally, I believe advancements across this discipline will be driven by successful start-up companies and drive advancements in hardware and software, eventually working it's way to the enterprise.  This event was bursting at the seams with attendees and the content, although packed into an efficient half-day session, was relatively rich.  Finally, the speakers and panelists were impressive.  Throw that all together and it you get the sense that this is real versus imagined marketing hype.

4.  Is Big Data important for the Enterprise?

R:  An unqualified yes.  This is such a no-brainer to me because data has always been critical in the enterprise.  Based on the drivers of the Big Data movement, the enterprise will be dragged along with the advancements in hardware, software, etc.  In addition, firms have the opportunity to lead and differentiate based on adopting and developing Big Data capabilities.

What is written next is the single most important subtlety I recognized during the session.  For every enterprise Big Data effort, the focus should be on the desired outcome first.  Let’s face it, enterprise is not start-up.  It already exists and has customers, products, a market, and a balance sheet.  Even though costs have come down and capabilities have gone up in Big Data, it doesn't make sense for enterprises to pursue Big Data just as a matter of course.  A firm should lay out their desired outcomes and then work to determine if Big Data is, or can be part of, the way to get there:

• Do desired outcomes involve lots of data?
• Is there a need to establish an interactive dynamic dashboard or User Interface for this data? Why?
• What judgment needs to be applied to the data?
• What insights need to be derived from the data? For the firm? marketplace? customers? product offerings?
• Is firm strategy enabled by advancing Big Data capabilities?
• Will any combination of the above simplify efforts across the firm?
• Are we truly willing to do the heavy analysis required to pursue Big Data objectives? (E.g. understanding limitations of existing data infrastructure and finding ways to evolve, review Cloud strategies, determine Data Architecture needs and content ownership, etc.)

At the beginning of this post I mentioned that attending the session was absolutely worthwhile.  That’s because I came out of this event energized, looking forward to watching where Big Data will go as it continues to develop and evolve.  I also look forward to applying Big Data concepts as I consult and deliver for my clients.  At a minimum I suggest - stay tuned to Big Data.  I know I will.