The Big Data Challenge from an SMB Perspective

Big data is a hot topic throughout the IT and business industry. I’ve seen many new start-ups trying to capitalize on the needs of firms to store, manage, and derive value from large databases with both structured and unstructured data. High storage needs and even higher processing power are required to play in this game, driving the fastest growing job description to now include some sort of combination of “data” and “scientist”. Source. Even Harvard Business Review has touted this role as “The Sexiest Job of the 21^st Century”. Source.

But in the SMB space, big data can mean something very different, with a slightly different set of challenges. Last week at the Exact Macola Evolve 2014 conference, Scott and I met a number of small business owners / managers who faced challenges with their data structures and file sizes. One President of a manufacturing company said to me, “My CFO has this spreadsheet full of pivot tables and graphs, and it’s now 100MB. We cannot e-mail it, and it takes forever to open. I don’t think I can virtualize our workstations if this is what is going on…” Well the actual answer is that yes, that can be virtualized and yes, that file can be re-structured to smaller pieces so that it’s not as much of a burden on the system resources when it needs to be accessed, edited, saved, or sent around. And the real answer is that there is a way out of this predicament. By and large, this is what Big Data actually looks like to the SMB.

Moving large files is not exclusive to the SMB, but the infrastructure to allow for ease of transfer isn’t generally there. To escape the clutches of e-mail transfers, some SMBs look to inexpensive storage/retrieval tools such as ShareFile or DropBox in order to collaborate. While these tools get the job done, some come with access hiccups while others are blatant security risks. And dividing up data into smaller pieces that can be updated consistently is easier said than done, easier planned for than retro-fitted. Alternatively, SMBs can invest in higher performance network hardware (think Ciena), but these have large pricetags associated with them. And although this is a generalization, the percentage of SMBs of the total number of businesses in the area increase as you move further and further out of major metropolitan areas. Another complicating factor as you move towards more rural areas is lower speed internet connectivity. Still, the SMBs face these “big data” challenges and they can consume many resources in determining how to deal with them.

Over the years, ManageOps has learned a thing or two about how to deal with these challenges. Here are some helpful tips when it comes to managing your SMB information:

1. Use a collaboration tool to keep large files OUT of e-mail / inboxes, insuring the tool does not increase the security risks noted earlier
2. Have reference files as read-only and store them in a protected area to maintain integrity (e.g. a protected network file server)
3. Create separate files for each type of analysis
4. Create graphs in separate files as they are graphically intense and the program will have to re-calculate and render on every edit, or every open

In a future article, I will write about the impacts of business intelligence being stored in employee Inboxes, instead of the tools designed to store and harness information. They are significant, and are felt across organizations of all sizes, so stay tuned.