Ability to injest Syslog and other Log formats
complete
J
James Stull
With the SIEM, I would really like to see the ability to injest logs from syslog sources and from various devices. Such as Ubiquiti, pfSense, fortigate, and other firewalls/switches. Also other devices would be good such as printers and IoT devices.
While these can generate a lot of noise, if we know how to best configure we can filter the noise out prior to shipping them to you.
Chris Bisnett
complete
With today's launch of the Managed SIEM Public Beta, we are able to ingest Syslog data using the existing Huntress Agent on endpoints you designate. That data will then be split out to the unique Syslog data sources and will be parsed and tracked based on those sources.
There is still more we will be adding to our Syslog support in the coming weeks including a public IP address that can receive Syslog data and better support for more sources. Today if we receive Syslog data for an unknown source or one we don't support we will store the data and will do some minimal parsing to index the data so that it can be searched. We'll use the additional feedback requests to track interest in additional Syslog source types.
R
Ryan Reinsch
Chris Bisnett This is GREAT news. Do we just start sending our Syslog to a designated agent and the rest will be dealt with on the backend? Is there any requirements of the Agent we send the Syslog data to?
Chris Bisnett
in progress
We've started work on Syslog ingestion. This will allow us to ingest logs from any device or application that can send Syslog formatted event data to a Syslog collector. Initially we will add this functionality into our Huntress agent to collect these logs from the local network. In the future we may also add publicly accessible Syslog collection endpoints for setups that have the ability to support encryption and where it is easier to not have to run a local agent for collection.
We will parse this data into the standard ECS format just like we do for Windows Event logs. This way there will be common fields that can be searched and will return different types of log data when the field matches.
We are looking for folks who are interested to help us test this in the next week or two.
J
James Stull
Chris Bisnett any perticular devices/brands you are looking to start with?
J
Jason Cook
Chris Bisnett We are fortigate if you need a fortigate partner to help
S
Scott Thomson
Chris Bisnett Hi - we have a few sources of possible syslog data including one that I'm super curious can feasibly be implemented, but I'm a bit hesitant to overshare in public. Could you reach me by email so we can discuss? Would love to participate, but not sure if it makes sense at this stage. Thanks!
I
Ilyas Esmail
Chris Bisnett we are available to test this, most of our clients use UniFi if we can send over logs from there.
J
Jason Walter
Chris Bisnett We would like to test.
R
Ryan Reinsch
Chris Bisnett We can test if this is still an option. We are a Fortinet shop.
Chris Bisnett
planned
Ingesting Syslog data is planned to start in the next week or two. While this seems pretty easy on the surface, when you dig in you quickly realize that all of the different network devices and systems that can send logs to a Syslog endpoint all do it differently and support various configuration settings, it quickly becomes clear this will initially support some systems and will iterate over the next few months.
Obviously dropping as much of the noisy data at the source is the best option, but in cases where that can't be setup, we plan on being able to drop noisy data at the ingest site and this won't count against the log volume.
J
James Stull
Chris Bisnett Awesome plan.
Yup, no two log sources are the same. I'm not sure if you looked at it or not, but you may want give the open source project graylog a look at. They can ingest logs on just about anything. It may have some ways that can help shorten your timelines.
Honestly, what I think it the hardest part is going to be the filtering. I would think a lot of that will depend on us in some ways unless you can filter it out on a per vendor basis. But even then I bet a lot of huntress clients will just ship everything at you in order not to miss anything, I could see the filtering getting overwhelmed in larger deployments.
In short, this is not an easy project.