Risk Focus is an expert implementation partner of Splunk in the financial services arena and a big proponent of the value Splunk delivers. We have implemented Splunk in numerous banks, funds and clearing/exchange firms across multiple mission-critical use cases.
Yet Splunk is not the only log analyzer out there. In particular, ELK (elasticsearch + logstash + kibana) is a growing open source offering which purports to have the same functionality as Splunk at virtually no cost.
Is this a valid claim? Do Splunk and ELK provide similar functionality, stability and technical richness needed by corporate institutions like banks, asset managers, hedge funds, exchanges, industry utilities and major technology providers?
Cost of Splunk vs. ELK
Let’s start by taking a look at the overall cost of investing in Splunk versus ELK. Corporate buyers invariably look at total cost of ownership (TCO) when making their buying decision because they want to know the true “all in cost” not just the up front fee.
“Splunk is expensive and ELK is free.”
A web search will turn up hundreds of blog entries claiming Splunk’s pay-per-gigabyte-indexed pricing model is expensive. Splunk data indexing charges sound pricey, but the way the pricing actually works is far cheaper than it first appears. Yes, an ELK license is free but Splunk is amazingly cheap, too.
Underlying the “Splunk is expensive” claim is the assumption that all data will be indexed, which is rarely true. A proper implementation includes an up-front analysis such that only the valuable subset of a company’s data is indexed.
Midsize and larger companies tend to purchase software and data licenses at bulk discounted rates. This gives a discount off list price and provides predictability (no “bait and switch” surprises) after adoption. For less than the cost of a single skilled FTE in a G10 country you can index a huge amount of log data with Splunk across your IT infrastructure and earn tremendous efficiency cost savings. We’ve seen Splunk’s rates dropping over time, so it’s getting even cheaper. If you just need to dabble, do basic development and testing, or a proof of concept, Splunk offers a free Enterprise license up to 500MB per day.
The primary concern for sophisticated corporate buyers is cost-to-value or total cost of ownership (TCO). Data license costs (at least at the pricing level of Splunk or other log analyzers) tend to be the least important factor in the TCO equation. The value of cost savings and new revenue discoveries can provide immense financial value that dwarf the license costs.
“Once we get ELK going it will be cheaper. The time to learn and get ELK figured out is an investment.”
Both Splunk and ELK can be installed and running quickly at a basic level with minimal learning time. Efficiency gains from reducing “Team GREP” activities (which never really identify or resolve underlying issues well), the actionable intelligence obtained (allowing you to optimize IT spend very quickly), and operational risk reductions all argue in favor of installing a log analyzer as soon as possible.
Once installed, the question then becomes “can Splunk or ELK knowledge can be rolled out through an organization quickly?” Rapidly enabling multiple users to take advantage of indexing, correlation and dashboarding capabilities is necessary to generate business value from a log analyzer.
Splunk is far ahead of ELK in speed of roll out and depth of coverage. Splunk offers a rich education program, a Professional Services group and an expansive network of skilled consulting partners. Getting a team Splunk certified takes less than 1 month. Hiring a Splunk partner firm to roll out capabilities quickly and build advanced correlation apps can further shortcut the time-to-value. Splunk has a large App Store with hundreds of free and paid apps to connect to standard IT hardware and software platforms. ELK is growing rapidly and is making similar education efforts, but is years behind Splunk in these critical areas.
“Beware the hidden costs.”
Compared to Splunk, we believe the investment required to institutionalize ELK is far more time-consuming and costly in terms of lost efficiencies and investment dollars than buying Splunk. An ELK user is likely to find that they must create an entire infrastructure around knowledge transfer, skill-building and connectors to underlying log sources before rolling ELK out across the firm.These are hidden but serious costs of choosing ELK. The incremental cost of a Splunk data license is much lower then the time and costs of building an in-house knowledge and support structure from scratch around ELK.
For a small organization with basic needs and a small IT support group, ELK is certainly a good investment. It’s free from a license standpoint and rewards DIY approaches. If you’re an IT development shop ELK is an excellent choice.
For large and sophisticated firms, especially financial institutions, energy companies, defense firms and the like, Splunk is cheaper in the long run.