The market for advanced analytics and the tools that support it continues to grow, creating a very lucrative space for brands that can gain and hold market share.
Whilst traditionally dominated by the big enterprise software vendors, particularly IBM and SAS, the last decade has seen rapid change in both the structure and visibility of this sector.
Most importantly, the emergence of really good, really cheap open source tools such as R (and its commercial version, Revolution R) and RapidMiner have challenged the assumption that enterprise analytics should be a costly exercise.
Whilst analysts – being a cautious lot – are not necessarily advocating freeware, open source tools and the community-generated nature of their features are a natural fit. Those working in the advanced analytics space are code-savvy types who are not averse to building their own features and creating hacks to assist in tricky problem solving.
Does the entry of solid open source tools spell the end of ‘easy money’ for incumbent enterprise analytics vendors? The jury is still out, but incumbents beware. High cost vendors will need to provide far more competitive pricing models and an exponentially greater degree of feature innovation – and fast – to prevent their market position being eroded by nimble, cost effective alternatives.
Open source analytic tools – and indeed open source tools in general – have some compelling advantages. Analytics uses a mix of data manipulation, statistical techniques, mathematical algorithms and graphical data representation – and commercial ‘closed source’ tools tend to be strong in one or two of these areas, but not in all.
By contrast, open source alternative R has packages that excel in each of these areas making it able to deliver a full toolkit for analytics. This depth of useful functionality is an artefact of continued development by the open source community who build modules to meet their ‘real’ needs, rather than the perceived needs conceptualised by a product manager. Coming from a wide developer base, open source tools draw on a broad pool of technical and commercial experience, which can lead to the creation of more elegant and useful solutions.
However, whilst capability depth makes open source analytic tools extremely powerful, they don’t spoon-feed, and thus require a greater level of user capability. If you can write code and know your way around statistical formulae, their performance and efficiency will win you over. If you can’t function without point and click, trying to switch to open source tools will be a rude awakening. This high technical bar is ironically a bit of a plus when selling in to an analytical audience – a tool that pushes the user to write elegant code and prove their capability is the type of thing that gets us geeks a bit excited.
Unsurprisingly, whilst open source solutions have been embraced by the education sector and small business, the corporate sector is more cautious.
Although 76 per cent of enterprises say they have embraced some open source tools, many still harbour reservations about shifting fully to software that has an open source code base, presenting rational sounding objections for keeping ‘traditional’ vendors’ tools in place as a backup. The most common objections are a trifecta of worry – security, stability and support.
Lets start with the most obvious objection from a risk management point of view – support. A frequently voiced objection to open source analytic tools revolves around the distributed nature of open source development, – most specifically the lack of a product ‘owner’ if things go wrong. Not having a vendor to throw the book at when problems occur makes operational managers rather nervous. This is a valid and very sensible concern to raise when choosing analytic software. Whilst open source communities tend to work very well to solve user problems, they do so in their own time – which isn’t helpful when the problem in question is urgent.
For this reason, commercialised ‘paid support’ providers have emerged for many of the open source analytic products, including brands such as Revolution Analytics who push a version of R with stabilised regular releases and a committed support program. This ‘back up’ of skilled support makes the underlying open source products more appealing to organisations with severe risk aversion, as well as those validly looking to use open source solutions in business critical processes.
The fear of ‘buggy’ releases causing mission critical systems failing is a lever enterprise software vendors can – and sometimes do – lean on to push the value of an established solution. And they have a point, in that completely free and unregulated open source tools can be hard to work with in a commercial setting, and can fail if not closely supervised. Community driven development can mean tangential shifts in functionality, and inconsistent release schedules that don’t gel with regulated business process.
But this is only true of less mature open source developments. Stable established tools with a strong user community rarely show ‘catastrophic failure’ – and, lets be blunt, we all know cases where high cost commercial solutions have fallen over as well.
For those situations when ‘always on’ reliability is essential, there are an increasing number of reasonably priced open source products originated and overseen by commercial developers that provide the best of both worlds – continued innovation from the user community and affordable pricing, paired with an organised test and release process from a commercial vendor.
The other big concern voiced by businesses is that due to the public nature of its code base, open source isn’t secure. Whilst again, there is a grain of truth to this, any security vulnerabilities in open source systems can easily be countered through appropriate IT security.
Open source tools are secure enough that they are used by the CIA and NSA– the latter of which has also open sourced its own NoSQL database system. And this approach to development may be the way fo the future – at least, it has enough potential that the Pentagon’s Defense Advanced Research Projects Agency (DARPA) is currently funding over 70 projects in open source data infrastructure, analytics and data visualisation. If the security of open source analytic tools isn’t a showstopper for the world’s most paranoid and security conscious organisations, then it’s probably not an issue the rest of us should get too worried about, relative to other security risks.
Will open source trump enterprise solutions?
Open source still has a hurdle of executive scepticism to overcome in order to become a universally preferred option, but as more examples emerge of ‘big business’ using open source tools with success, the open source proposition looks increasingly compelling.
Lower cost, wider functionality, more flexibility and a strong user community – all these traits point to a much stronger ROI and faster payback than currently seen for high cost analytic platforms.
Of course, a smart counter play by enterprise vendors could change this trajectory. Moving to low cost SAAS or highly differentiated tiered pricing options whilst still a majority preference could bode well for the retention of analytic software contracts with more progressive organisations – provided it does not come at the expense of features and stability. And there will always be change-averse buyers who wait for early adopters to iron out the kinks in new platforms and approaches, easing the revenue impact of early adopter preference for alternate solutions.
Perhaps enterprise analytics vendors can find insight in the experience of vendors in the data management space, where open source and enterprise have found a kind of equilibrium. Smart vendors have reacted effectively to open source by embracing it and producing new low cost feature rich tools such as Amazon’s Redshift and Google’s Big Query, or release minimal cost versions of well-established enterprise tools such as SQL Server Express.
Enterprise analytics vendors need to do this too – or face the realistic risk that future generations of analysts will have no exposure to or interest in using their tools.