Syslog dashboards

Syslog Data

Syslog is a widely used standard protocol for logging system events, errors, and other messages in computer systems. Syslog data can be incredibly valuable for detecting security breaches, identifying system errors, and tracking down performance issues. However, working with syslog data can be challenging, especially when dealing with large amounts of data. In this blog article, we'll explore the two main ways to work with syslog data: parsing the data and working with the raw data.

syslog man

Parsing the syslog data

One way to work with syslog data is to parse it and create dashboards with real-time analytics. Parsing the data involves breaking it down into its constituent parts, such as the timestamp, source IP address, message type, and severity level. This parsed data can then be stored in a database, and visualized in dashboards using tools like Viewtinet.

The benefits of parsing syslog data include:

  1. Real-time analytics: Dashboards can be created that provide real-time insights into system performance, security breaches, and other critical events.

  2. Better searchability: Parsed data can be easily searched and filtered, making it easier to identify specific events or patterns in the data.

  3. Reduced storage costs: Parsing data can often result in a smaller data footprint, which can help reduce storage costs.

Syslog dashboards

Working with syslog raw data

Another way to work with syslog data is to work with the raw data. In this case, the analytics tool needs to understand the different available values per log. Raw data is the unprocessed data that comes directly from the syslog source, without any parsing or manipulation.

The benefits of working with raw syslog data include:

  1. More flexibility: Raw data can be transformed and analyzed in a variety of ways, depending on the needs of the organization. This flexibility allows for more customized analytics and reporting.

  2. Deeper insights: Working with raw data allows for deeper insights into system performance, security breaches, and other critical events. This is because raw data can provide more detailed information than parsed data.

  3. More comprehensive coverage: Working with raw data can help organizations capture and analyze all available data, rather than just a subset of parsed data.

Raw syslog data

Which approach is best for syslog?

Deciding which approach to use depends on the needs of the organization. If real-time analytics and reduced storage costs are important, then parsing the data may be the best option. On the other hand, if flexibility and deep insights are more important, then working with the raw data may be the better choice.

Ultimately, it's important to have a solid understanding of the organization's needs and goals when working with syslog. By doing so, organizations can make more informed decisions about how to best work with their syslog data and achieve their desired outcomes.

 

Despite this, there may be some environments where the amount and complexity of data are so vast that the query response time may be too long. For these cases, the index logging will solve any performance issue. As a result of the indexing, you might not be able to drill down or follow different post-analysis troubleshooting. Nevertheless, if you already set the parameters that you want to analyze and correlate, index logging is the best method to get real-time results.

Using Viewtinet for syslog

The vision of Viewtinet has always been to create a flexible platform suitable for several and different use cases. Having said that, why limiting to one method when you can have both at the same time?

Viewtinet proposes an end-to-end solution. It is capable of integrating the data, store it and provide full analytics and observability. Thanks to the visual smart data broker, Viewtinet can integrate and parse syslog data from any vendor.

However, depending on the vendor and type of syslog data, or even if the company needs to keep the raw syslog data for regulatory reasons, Viewtinet can also work in this format. It has an intelligent layer to identify the different keys and we can select a key to identifiy the logs that content that key, and then filter by the available values.

Viewtinet chosen as a 2021 Top 100 Startup Winner

As 2021 comes to an end, it is time to look backward and analyze the year earns. We all started the year with optimism after the unusual previous 2020 but with a lot of doubts about how the pandemic was going to behave. We gave total freedom to work from home and we preferred to avoid face-to-face meetings and travels. This could be a handicap for a startup but the conclusion is that we have to be very proud of our team. The product improvements have been outstanding and the amount and size of companies that are trusting Viewtinet’s team and technology are increasing.

All this effort and work has had a very nice surprise before the year ends: Red Herring has chosen Viewtinet as a 2021 Top 100 winner in Europe.

Press Release

December 16th, 2021; - After much consideration, the Red Herring judging panel has announced its Top 100 Europe winners this evening, recognizing Europe’s leading private companies and celebrating these startups’ innovations and technologies across their respective industries.

Red Herring Top 100 Europe enlists outstanding entrepreneurs and promising companies. It selects the award winners from approximately 1,200 privately financed companies each year in the European Region. Since 1996, Red Herring has kept tabs on these up-and-comers. Red Herring’s Top 100 list has become a mark of distinction for identifying promising new companies and entrepreneurs. Red Herring’s editors were among the first to recognize that companies such as Alibaba, Facebook, Google, SalesForce.com, Skype, SuperCell, Spotify, Twitter, and YouTube would change the way we live and work. Thousands of the most interesting and innovative companies have graced the Top 100 list over the years.

“In 2021, selecting the top achievers was by no means a small feat,” said Alex Vieux, publisher and CEO of Red Herring. “In fact, we had the toughest time in years because so many entrepreneurs had crossed significant milestones so early in the European tech ecosystem. But after much thought, rigorous contemplation and discussion, we narrowed our list down from hundreds of candidates from across Europe to the Top 100 Winners. We believe Viewtinet embodies the vision, drive and innovation that define a successful entrepreneurial venture. Viewtinet should be proud of its accomplishment, as the competition was very strong.”

Red Herring’s editorial staff evaluated the companies on both quantitative and qualitative criteria, such as financial performance, technological innovation, management quality, overall business strategy and market penetration. This assessment was complemented by a review of the track records and standings of similar startups in the same verticals, allowing Red Herring to see past the “hype” and make the list a valuable instrument of discovery and advocacy for the most promising new business models in Europe.

Following Viewtinet’s well-deserved win, they are then invited to showcase their company to the US market at the Top 100 North America event and compete internationally for the Top 100 Global in October. Red Herring is dedicated to following Viewtinet’s path to further success and innovation.

Dashboards

Index logging versus index-free logging

When searching for a log management solution, some providers are claiming the advantages of index logging, and others claiming the advantages of index-free logging. Which one is the best? In my opinion, it depends on different factors. In this article, I will try to bring some light to this.  

Index logging

With this method, the data is being processed upfront. It creates different tables that combine different indexing options. As a result, when querying the database, the result will appear instantly. Moreover, even when managing a vast amount of data, the index-logging leads to instant query results. Certainly, this method will induce a high-speed response. However, we must take into consideration that this upfront processing needs more CPU & RAM from the server. In addition, the servers need more disk space because of the tables created by the indexing. Also, you need to define upfront the searchable elements and/or correlations.

Index-free logging

In contrast, with index-free logging, the data is stored as a table on the fly. The ingestion process is quicker and does not consume additional CPU, RAM, or disk space. Furthermore, there are several mechanisms to speed up the queries results: containers, data compression & data aggregation. On the whole, thanks to the current state of the art, this method can provide real-time responses to heavy queries. Undoubtedly, the results will not appear at the same speed as index logging, but still, it seems likely that this experience is enough in most cases. Moreover, everything is searchable.
Dashboards

Conclusion

I’d say that depending on the use cases, the complexity of the data sets, and their sizes, the optimal approach could be one or the other. Evidently, index-free logging is less costly in terms of hardware. Not only it can ingest data faster with fewer resources but also unindexed tables can provide deeper analytics procedures. What I mean is that when you are investigating an issue, with unindexed tables you can drill down by any of the available dimensions.  So, if you are analyzing stored data, it seems like the best approach. Despite this, there may be some environments where the amount and complexity of data are so vast that the query response time may be too long. For these cases, the index logging will solve any performance issue. As a result of the indexing, you might not be able to drill down or follow different post-analysis troubleshooting. Nevertheless, if you already set the parameters that you want to analyze and correlate, index logging is the best method to get real-time results.

The Viewtinet approach

The vision of Viewtinet has always been to create a flexible platform suitable for several and different use cases. Having said that, why limiting to one logging method when you can have both? The Viewtinet style is to be easy to use and intuitive. Viewtinet has already integrations with most of the IT data sources in the market. The templates in the Visual Smart Data Broker have already taken into consideration the optimal logging method. So from the user experience, Viewtinet is already providing the most efficient configuration. With new or custom data sources, what Viewtinet does by default is index-free logging. This way, the system administrators can benefit from having all the data stored, to create any report, dashboard, and to drill down. After that, in case a report or several reports appear with some delay, with the visual interface, you can click the indexing option to switch from index-free logging to index-logging. This way, you may not know upfront how you are going to analyze the data. You will have all the possibilities. Then, once you design your dashboard you can optimize the performance by indexing specific reports if needed.