Humio Implementation
Best Practices
Based on Vijilan’s proven approach to log analytics and security operations
Abstract:
Humio provides a solution for effective log management. It enables real time, large-scale log analytics, a necessity for Security Operations (SecOps), and DevOps. Vijilan, a longtime Humio partner, offers a unique Humio implementation process based on best practices and its own proprietary technology developed to make Humio work optimally. This paper explores these best practices and highlights what they mean for a SecOps and DevOps team who want to improve their logs collection, log storage and log analytics. While this paper focuses on SecOps, the methodology for logging and storing log in Humio for DevOps is performed in a similar fashion.
Introduction
Effective log management is a critical, though often elusive, element of Security Operations (SecOps). Successful incident response is impossible without it. However, getting on top of logs from multiple devices, users, applications and other log sources can be a significant challenge for a SecOps team. Humio offers a solution, one that delivers real time, large-scale log analytics.
Making Humio work requires a focused, meticulous implementation. Vijilan, a longtime Humio partner, has developed a proven set of best practices for Humio implementation based on numerous client engagements over the last several years. Vijilan augments these practices with its own proprietary technology. This paper explores Vijilan’s Humio implementation best practices and highlights what they mean for a SecOps team that wants to improve its log analytics and incident response capabilities.
Overview
SecOps is responsible for the operational side of defending an organization’s digital assets against malicious actors. In practice, this means detecting and responding to threats quickly enough to prevent negative outcomes. A SecOps team will be doing its job well if it can, for example, detect a ransomware attack and respond by isolating the malware before it encrypts any valuable data.
Successful SecOps involves the rapid, simultaneous analysis of enormous amounts of data from multiple sources. In particular, the SecOps team must run advanced analytics on logs from multiple systems, including security tools like intrusion protection systems (IPS’s) and firewalls, as well as operational systems like network switches and identity and access management (IAM) systems. With today’s highly sophisticated attackers, the evidence of a threat can easily be hiding in plain sight amongst a deluge of log data. Having the ability to correlate telemetry data from several security products on the fly allows SOC analysts to find therats quicker than performing it manually If SecOps can detect the threat quickly enough, it can prevent a breach.
Log management, analytics and SecOps
Log management challenges in SecOps
Log management presents a number of challenges to SecOps teams. In terms of requirements, log management demand fast ingestion of large amounts of log data from a variety of sources at great velocity. This is easier said than done, and some solutions require the SecOps team to set up server clusters to manage growth in log data sources. As log data volume grows—and it usually grows exponentially—it may also be necessary to deploy indexing farms. Storage requirements balloon in the process, too. Performance can lag, which may negatively affect the security analysis that is the point of the whole endeavor. SecOps may need to assign engineers just to the work of managing and updating the infrastructure that supports log management.
Humio:
Solving log management challenges
Humio is a modern log management platform designed to process the scale and complexity of today’s log management workloads. Humio is designed with two key differentiators which make real-time analytics at scale possible: Data-streaming in an index-free architecture and high compression storage. These two factors combine to enable Humio users to ask anything and get instant responses from log data.
Best practices for Humio implementation
Vijilan has assessed and validated Humio as a log management solution for applications and security for over four hundred organizations in finance, healthcare, government, manufacturing, legal and education. A growing set of best practices has emerged from these experiences. When Humio is deployed in accordance with best practices, the client enjoys optimal usage of Humio, in terms of SecOps results, utilization of IT assets and team productivity.
Some of the best practices described here are based on a security event management web application that Vijilan has developed for Humio. The application enables users to better deal with real-time monitoring, correlation of events, notifications, visualizations, and presentation. It can correlate sequences of events, traps, logs and metrics across multiple dimensions directly from Humio.
Vijilan uses Humio for long-term storage, analysis, and reporting of log data, leveraging Humio as a Security Information and Event Management (SIEM) solution with advanced analytical capabilities. Through this partnership, Vijilan can provide a complete, end-to-end SIEM with log collection, parsing and normalization, real-time detection, analysis, and reporting.
Humio can run ultra-fast searches and queries against raw log data in seconds and presenting the information visually to users. The query window generates results from inputs ranging from a simple text-based search to a multi-variable structured query of billions of records within seconds.
Core processes
Best practices for Humio integration start with a series of core processes:
- Vijilan has successfully developed its own Security Event Management to better deal with real-time monitoring, correlation of events, notifications, visualizations and presentation.
- Design a log analytics dashboard in alignment with SecOps staffing and workflows
- Determine alerts and align with SecOps staffing and workflows
- Operationalize log collection
- Document Humio log analytics solution for specific SecOps parameters
Some of these are not directly related to log management and analysis. The broader SecOps context matters for getting log management and analysis right. Humio implementers should understand where log management fits into the SecOps team’s mandate and obligations, e.g., if real time ransomware detection is a priority, then decisions about which logs to integrate into Humio ought to align with that priority. In this way, SecOps’ strategy and goals form a baseline for the Humio implementation project.
Humio:
Determine log integration requirements: current and future
The Vijilan approach to Humio implementation includes the development of an architecture definition that includes project requirements. As part of this architecture definition, the Humio implementation team must determine log integration requirements: which logs will be integrated? What connectors and other technologies will be needed? What are the parameters of the log data parsing and normalization?
This thought process and development of requirements should apply to current and future log integrations alike. It may be difficult to know future requirements for sure but thinking through likely scenarios for future integration is a best practice, one that will pay off as the Humio instance inevitably evolves over time.
Operationalize log collection
The implementation process continues with the setup of log collection. This will involve the deployment of threat sensors and cloud connectors. At the functional level, what is happening is the creation of a data lake into which a huge, diverse amount of log data will flow. Making this work means ingesting and normalizing all the various log data streams and then “shipping” them into Humio. Vijilan has a pre-built solution for on-premises and cloud log collection.
A best practice at this stage is to collect historical log data in addition to real time and recent data flows. Forensic analysis of old logs can reveal critical information about past attacks, which may have not been detected. Having access to historical log data in Humio can also facilitate predictive threat detection going forward.
Determine alerts and align with SecOps staffing and workflows
Vijilan collaborates closely with clients on determining alerts for Humio users. It is essential that alerts align with SecOps staffing and workflows. For any alert, there needs to be a SecOps team member to deal with it. or the alert needs to feed into an automated incident response system, such as ITSM solution. This matters because alert fatigue is a major issue in SecOps. Flooding the team with alerts is never optimal. Indeed, the practice can cause important threats to be missed. The best practice is to map the staffing and workflows carefully and design alerts to match those parameters.
Design a log analytics dashboard in alignment with SecOps staffing and workflows
Visualization of Humio log management is a key to SecOps effectiveness and team productivity. Vijilan works with Humio implementation clients to design and deploy customized dashboards. The best practice is to follow a dashboard development process that includes widget design, mapping of log sources, validation, and documentation.
Document Humio log analytics solution for specific SecOps parameters
Documentation tends to be neglected in IT projects, but that does not make it any less important. A Humio implementation must be documented to some extent. Because Humio already offers documentation of its solution, there is no need to draft a big book to go along with the implementation. A succinct piece of documentation, which highlights the parameters of the implementation, will prove itself to be extremely valuable as the Humio instance expands and evolves.
Proactive services
As the Humio implementation comes to life, Vijilan engages in further proactive services. Each of the following could be considered a best practice for getting the most out of Humio:
Run test cases and provide feedback to SecOps team
Identify and train key personnel
Assess the suitability of outsourcing log monitoring
Vijlian’s toolset offers a range of security analytics capabilities integrated with Humio’s functionality, including:
- Alerts
- Compliance reports
- Proactive threat hunting
- Investigation and incident Response
- Dashboards
- Executive reports
Run test cases and provide feedback to SecOps team
Setting up Humio is not a push button process. The solution needs tuning. The best practice, therefore, i is a continuous and iterative process with well-defined milestones and goals. As Humio processes the test cases, Vijilan provides feedback to the SecOps team. In this way, Vijilan can get Humio properly tuned while simultaneously mentoring the SecOps team on how to improve their log analytics and incident response processes.
Identify and train key personnel
People will need to make Humio work once it’s deployed. If the client does not already have staff trained for Humio, it’s a wise idea to use the implementation project period to identify and train key personnel. Management and admin for Humio does not have to be a full-time job. It can be part of someone’s role in SecOps. But someone will need to take responsibility for monitoring Humio and its supporting infrastructure.
Assess the suitability of outsourcing log monitoring
Outsourcing log monitoring can be a best practice for some organizations. The decision to engage with an external service provider for this workload may arise from a lack of personnel on staff or a desire to focus human resources elsewhere. Given the constraints on SecOps staff and the general difficulty in finding competent employees in this specialized work, letting an outsourced provider handle log monitoring may be a good move.
Conclusion
- Getting Humio to work effectively means following best practices. With Vijilan as an implementation partner, an organization can realize its goals for real time, large-scale log analytics. SecOps will improve as a result.
- Best practices include determining log integration requirements for the present and the future, operationalizing log collection and solving the problem of “log shipping” and determining alerts and aligning them with SecOps staffing and workflows. A well-designed dashboard is essential, as is the process of documentation.
- Vijilan offers its own unique, proprietary toolset to facilitate the implementation in accordance with these best practices, and others, such as running test cases and training key personnel. As these factors come together, a successful Humio implementation will be the result.
Conclusion
Getting Humio to work effectively means following best practices. With Vijilan as an implementation partner, an organization can realize its goals for real time, large-scale log analytics. SecOps will improve as a result. Best practices include determining log integration requirements for the present and the future, operationalizing log collection and solving the problem of “log shipping” and determining alerts and aligning them with SecOps staffing and workflows. A well-designed dashboard is essential, as is the process of documentation. Vijilan offers its own unique, proprietary toolset to facilitate the implementation in accordance with these best practices, and others, such as running test cases and training key personnel. As these factors come together, a successful Humio implementation will be the result.
About Humio
Humio is a modern log management platform that is purpose-built for today’s complex systems and scale. Humio is built with two key differentiators which make real-time analytics at scale possible: Data-streaming and index-free architecture and high compression storage. This enables customers to ask anything and get instant responses. The Humio platform also has a robust ecosystem that integrates with technologies (such as Vijilan) that will allow Humio to be used as a security solution among large-scale enterprise and education institutions.
About Humio
Humio is a modern log management platform that is purpose-built for today’s complex systems and scale. Humio is built with two key differentiators which make real-time analytics at scale possible: Data-streaming and index-free architecture and high compression storage. This enables customers to ask anything and get instant responses. The Humio platform also has a robust ecosystem that integrates with technologies (such as Vijilan) that will allow Humio to be used as a security solution among large-scale enterprise and education institutions.
About Vijilan
Vijilan Security has over fifteen years of experience specializing in monitoring, detecting, and responding to information security incidents. Vijilan is a US-based Limited Liability Company found in Aventura, Florida. Vijilan’s primary Security Operations Centers (SOCs) run 24/7, collecting events from private and public networks globally. Vijilan operates and stores all US-based customers’ information in the USA. This company is led by a team of engineers, developers, and highly skilled information security professionals, and services more than 500 small and medium businesses (SMB), focused on Banking, Health Care, Government and Education, in the US, Australia, South Africa, Brazil, and the UK.
Vijilan has specialized in creating the security analytics necessary to collect, analyze and filter out relevant information from multiple verticals. Having broad visibility into hundreds of distinct and isolated networks has given Vijilan the ability to develop complex sequential and non-sequential correlations rules to detect malicious insiders and outsiders within minutes. With its unique, comprehensive cloud-based security incident response application, Vijilan can quickly detect, triage, and escalate to proper levels for immediate response. Additionally, Vijilan’s elastic, cloud-based architecture offers high availability, uptime and scalability, and seamless integration with virtually any on-premises and cloud application.
What our clients say
Solid PARTNER in our security team! Reliable security software and SOC team! Vijilan has excellent reporting features to demonstrate value.
Troy Newman Cyber5
Vijilan has been a great partner and has really helped us increase information security with our clients... having a partner that allows us to provide a great service at a great price is invaluable.
Toby HigginbothamRegion 18
Vijilan has enabled us to start building out our MSSP practice. One of the steps to building out a successful security team is to have a SOC. Vijilan has made it possible, and while we don’t want to tell you all our secrets, this one is clearly out of the bag now. What a great vendor to do business with.
James RockerNerds That CareNerds That Care
We hired Vijilan to handle our cloud migration from start to finish. Vijilan collaborated extensively with us as a trusted advisor to address the migration's tactical problems, which included architecting, testing, and updating all resources in a live production environment. Vijilan’s methodology and architecture addressed long-term goals of scalability, high availability, and a cost-effective operational environment. Their continued assistance is prompt and dependable.
NoelRegion 18
We offer a completely automated predictive analytics system that runs in the public cloud, employing machine learning and other advanced data mining techniques on a large scale. We wanted a partner who could not only assist us tactically but also offer us a strategic roadmap to ensure we got the most out of our large investment in the public cloud. Vijilan has obviously exhibited an exceptional combination of technical skills, business savvy, and industry expertise. We will continue to rely on Vijilan for reliable advice.
Tyson Sdbworld
Great SOC and sales team.
I like the deployment and peace of mind for my techs. Vijilan does all the heavy lifting for running a 24/7 Security Operations and manage SIEM for my organization. I don't have to purchase SIEM or set up a SOC. simple and fair pricing.
Rob P.CTO I like the deployment and peace of mind for my techs. Vijilan does all the heavy lifting for running a 24/7 Security Operations and manage SIEM for my organization. I don't have to purchase SIEM or set up a SOC. simple and fair pricing.
Overall it has been great, they have a great development road map and it is very affordable for a SIEM/SOC
We love the SOC function that allows us more time to complete other IT initiative, knowing that there is a SOC team managing the alerts
Simon C.vCIO We love the SOC function that allows us more time to complete other IT initiative, knowing that there is a SOC team managing the alerts
The support team is fantastic and responsive when needed, point is that we never need them.
The software (collector), dashboard, and reports are what is expected. Great system for keeping track of alerts, responding, and closing. Very flexible on the collection of logs
Verified Reviewer The software (collector), dashboard, and reports are what is expected. Great system for keeping track of alerts, responding, and closing. Very flexible on the collection of logs
Solid PARTNER in our security team!
Reliable security software and SOC team to support! Excellent reporting features to demonstrate value.
Troy N.CEO Reliable security software and SOC team to support! Excellent reporting features to demonstrate value.
The experience has been great. They work closely with their partners, take their feedback seriously and continuously listen to suggestions. They fill a very important role and niche in the cyber security stack for SMB and MSP partners
We love that Vijilan offers a price effective SEIM/SOC solution for our managed service clients. It allows us to offer a solution to problems (compliance, regulation, general security log monitoring) that can be very expense for SMBs.
Ben V.Project Manager We love that Vijilan offers a price effective SEIM/SOC solution for our managed service clients. It allows us to offer a solution to problems (compliance, regulation, general security log monitoring) that can be very expense for SMBs.
It's been a changing and ever-growing partnership. It's been great watching the product grow and improve over the years. Everyone I've interacted with is passionate about the software and want to ensure their customers are satisfied and understand everything they're getting and what's going on. The support and discussions we're able to have directly with the team is extremely valuable
I like the amount of detail in the tickets, so I know what the issue is/where I need to review with just a glance. It then goes into greater detail at the bottom in case I need more
Ian I.Managed Service 1 I like the amount of detail in the tickets, so I know what the issue is/where I need to review with just a glance. It then goes into greater detail at the bottom in case I need more
Vijilan has been a great partner and really helped us increase information security with our clients. Primarly in education there is not a lot of money and information security is often the last priority so having a partner that allows us to provide a great service and a great price is invaluable.
Vijilan has been a great partner from the very start. They listen to our needs and truly help and want us to be successful.
Toby H.Coordinator of Network Operations Vijilan has been a great partner from the very start. They listen to our needs and truly help and want us to be successful.
Very easy to use. Great tool for on the move.
Nothing to dislike about this product. Set up is easy and support is good.
Robert M.Security Analyst Nothing to dislike about this product. Set up is easy and support is good.
Vijilan has been a great partner and really helped us increase information security with our clients. Primarly in education there is not a lot of money and information security is often the last priority so having a partner that allows us to provide a great service and a great price is invaluable.
Vijilan has been a great partner from the very start. They listen to our needs and truly help and want us to be successful.
Toby H.Coordinator of Network Operations Vijilan has been a great partner from the very start. They listen to our needs and truly help and want us to be successful.
Continuously adds reports and features. Willing to work with partners to customize the need for the end customer. Willing to listen to partners to improve the product.
Michael P.Sr. Systems Analyst
Previous
Next
Share this page on:
Contact Information
Experience peace of mind with a reliable top tier security solution.
Need more information?
Need more information?
-
954-334-9988
-
https://www.linkedin.com/company/vijilan-security-llc/
-
info@vijilan.com
-
20803 Biscayne Blvd #302 - Aventura, Florida 33180