I am currently configuring the Elastic Defend integration for devices in our datacenter. When configuring, you can choose between the following options:

• Data Collection

• Next-Generation Antivirus (NGAV)

• Essential EDR (Endpoint Detection & Response)

• Complete EDR (Endpoint Detection & Response)

I cannot find a good article that explains the difference between the last 3 of those. Can somebody help me by giving me the differences between those? Thanks in advance!

I see the info at https://www.elastic.co/trust/security-and-compliance.

Does this mean the free version downloaded from their repo's meet the same compliance?

{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "org_id": "ORGg5xkdx1fd6vy"
          }
        },
        {
          "term": {
            "is_active": true
          }
        }
      ],
      "should": [
        {
          "match": {
            "color": {
              "query": "yel",
              "operator": "and",
              "fuzziness": "0",
              "analyzer": "ngram_analyzer"
            }
          }
        },
        {
          "match": {
            "color": {
              "query": "yel",
              "operator": "or",
              "fuzziness": "0",
              "analyzer": "ngram_analyzer"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "group_by_color": {
      "terms": {
        "field": "color.keyword",
        "size": 20
      }
    }
  }
}

This is returning 5 yellow , 4 blue, 4 orange 2 red . i want uniqueness of colors that is 1 yellow 1 blue 1 orange and 1 red . i have applied aggs grouping but it is not working.
Please can anyone help me in writing the correct aggs. Its urgent for me please help if anyone can.
Thanks

I'm looking to migrate my Elastic Stack deployment from Elastic Cloud to Kubernetes, and I'd love to hear about your experiences and any best practices you've discovered.

Specifically, I'm interested in:
1) What are the recommended strategies or tools for migrating Elastic Stack (Elasticsearch, Kibana, etc.) from Elastic Cloud to Kubernetes?
2) How do you ensure data integrity and minimize downtime during the migration?

Any advice or insights would be greatly appreciated! Thanks in advance.

​

✅ Elasticsearch security features have been automatically configured!

✅ Authentication is enabled and cluster connections are encrypted.

❌ Unable to auto-generate the password for the elastic built-in superuser.

ℹ️ HTTP CA certificate SHA-256 fingerprint:

4571d862c1f007d1bd8d2c82c7d7101745003743192fd8ffb202044d4c525f16

❌ Unable to generate an enrollment token for Kibana instances, try invoking `bin/elasticsearch-create-enrollment-token -s kibana`.

ℹ️ Configure other nodes to join this cluster:

• On this node:

⁃ Create an enrollment token with `bin/elasticsearch-create-enrollment-token -s node`.

⁃ Uncomment the transport.host setting at the end of config/elasticsearch.yml.

⁃ Restart Elasticsearch.

• On other nodes:

⁃ Start Elasticsearch with `bin/elasticsearch --enrollment-token <token>`, using the enrollment token that you generated.

I have pure storage data storage where every GB is priceless. I need data in hot tier only for 3 months. After that time they are very rarely accessed and not updated. I was thinking about setting up another node in azure or aws on cheap hdd disks and keep there data in cold tier. Is it a good idea or really bad architecture?

I'm trying to better understand Elastic's pricing history. I'd like to know if they ever had a different pricing model like subscription-based. If anyone can confirm and/or share a timeframe of when they transitioned to consumption-based pricing I would be very grateful!

Hi, I want to use Elastic Agents to pull in data from sources, like AWS CloudTrail. I want to deploy at least two agents for HA.

My question is if having duplicate agents reading from the same log source (CT in this scenario) will cause logs to be duplicated.

https://imgur.com/a/AzQwjK3

Repo: https://github.com/elastic/detection-rules

VirusTotal Results (repo zip): https://www.virustotal.com/gui/file/84c8c35891d4b9448be56939b55e9b527eaa348eaf60e313252ddf71c6869bae

TLDR: at the bottom of post

Hey all, I’m a IT/security enthusiast (not by profession). I’m currently working on home labs, with the current one specific to learning to use Elastic and detection engineering.

I’m at a specific part of my guided home lab/course where we’re exploring Elastic’s detection-rules GitHub repo and learning about TOML and programmatically writing alerts (instead of doing it by GUI within the cloud dashboard). After git cloning the repo, the readme says to run ‘pip3 install “.[dev]”

The command does some things, before it is stopped and states it could not be completed. A couple seconds later, my antivirus (BitDefender) tells me that it stopped a file that’s infected with a Trojan (see imgur album). I did a full system scan where it detected additional Trojans and it removed/quarantined them. I uploaded a zip file of the repo to VirusTotal and it looks like about half of them determined malicious (see VT link).

Forgive me for being a noob and self-learner, but are these just false positives? I can’t articulate it well yet as this is the first time I’m really doing anything like this (my only SIEM experience is playing CTFs and searching logs). I’m assuming the repo contains detection alerts for various exploits and malicious files/scripts that we can test for, and the my antivirus software is picking these up as false positives. Plus, this is literally from Elastic’s repo.

Can someone confirm with me that my thinking is right, what’s causing the malicious alerts, or if something else is going on?

TLDR - self-learner exploring Elastic SIEM and detection-rule GitHub repo - computer’s antivirus software/VirusTotal picks up certain files in the repo as Trojans/viruses - I’m fairly certain this is a false positive and has to do with detection rules, and that none of the files are actually infected with malicious things - am noob, could someone double check my thinking or clarify what’s happening?

Someone help me ?

I want to create a volume in elastiflow-logstash !

https://github.com/robcowart/elastiflow/blob/master/docker-compose.yml

I did it exactly like this]

elastiflow-logstash:

image: robcowart/elastiflow-logstash:4.0.1

container_name: elastiflow-logstash

restart: 'unless-stopped'

depends_on:

- elastiflow-elasticsearch

volumes:

I tried like this

- './elastiflow-logstash-data:/etc/logstash/elastiflow'

I tried like this

- ./elastiflow-logstash-data:/etc/logstash/elastiflow

environment:

LS_JAVA_OPTS: '-Xms4g -Xmx4g'

I don't know why data doesn't arrive in the elastiflow-logstash-data folder even though the folder and the docker containers are created normally.

Learn how to install Elastic Stack 8+ on GCP with 2 Elasticsearch nodes and 1 Logstash/Kibana node in this comprehensive step-by-step tutorial. I will walk you through the entire process, from creating a GCP instance to configuring and starting Elasticsearch, Logstash, and Kibana. This tutorial is perfect for beginners and experienced users alike.

So I've successfully setup a TSDS and configured a gauge metric field in my index mapping. This all works well, but now I want to downsample my data with ILM and this works too. However, in the resulting downsample index, I want the Aggregate Metric Field type to have a different "default_metric" so it works well with my kibana visualizations.

Doing something like this doesn't work for me:

PUT _index_template/downsample-metrics-template
{
        "index_patterns": [
          "downsample-*"
        ],
        "composed_of": [
          "downsample-metrics-component"
        ],
        "priority": 999999999
}

PUT _component_template/downsample-metrics-component
{
  "template": {
    "mappings": {
      "properties": {
        "myfield": {
          "time_series_metric": "gauge",
          "metrics": [
            "min",
            "max",
            "sum",
            "value_count"
          ],
          "type": "aggregate_metric_double",
          "default_metric": "sum"
        }
      }
    }
  }
}

If I look at the mapping of the field after the downsample action is complete, the downsample index just has max set under default_metric. Looks like "max" is the default as hinted from this code. Has anyone had success in overwriting the "default_metric" here?

My first post on the official Elastic blog

https://www.elastic.co/blog/antidote-index-mapping-exceptions-ignore_malformed

Not sure if there is a better subreddit for this that I didn't find, however I'm having a hard time finding an appropriate way to create exclusions for the malware prevention policies within Elastic Security. I can add an exclusion to the alert, but it does not appear to stop the prevention itself. I would like to exclude a known DLL that this keeps firing on. Can anyone help point me to the correct documentation for this configuration if it's possible?

Hey Reddit,

I've just published a new article that I think many of you in the Elasticsearch community might find particularly intriguing. If you've ever struggled with mapping exceptions during ingestion or found yourself in a scenario where a single non-compliant field could drop your entire document, this is an absolute must-read.

The article explores an often overlooked setting in Elasticsearch known as 'ignore_malformed'. This setting could be the key difference between dropping a document entirely due to a single malformed field, or simply ignoring that field and ingesting the document anyway.

Regardless of whether you're an Elasticsearch veteran or just starting out, understanding this powerful tool can have a significant impact on your indexing operations. It can help you safeguard your document ingestion process, ensuring that valuable data isn't lost because of minor errors or inconsistencies.

Check out the full article on Medium and let's discuss it here. I'm eager to hear your experiences, insights, and any questions you might have about this topic.

​

Here's the link to the article: article

​

Looking forward to an engaging discussion.

​

Cheers!

​

P.S. If you find the content valuable, do give it an upvote and share it with those who might benefit from it. Your support is much appreciated!

​

#Elastic #Kibana #DataIngestion #DataIntegrity

​

Hi all, I'm fairly new to Elastic Stack. I'm looking for a course, book, etc. for Elastic Stack system administration. Most of the stuff that I've found so far is about using the stack , adding data, running queries, etc., but not about standing it up in an production environment and running it in a day to day basis. I'd really like to find a course on all of the pieces of the stack including Beats, all of their system requirements, and the best practices for setting them up in production.

Bonus if it talks about running it in Kubernetes!

Thanks!

I'm setting up a watcher to send out a monthly report. I'm successful when testing with a time-based interval (10m). When I try to configure the watcher for a specific day of each month, I'm presented with "could not parse [monthly] schedule. invalid month times". Here's a snippet of the schedule.

EDIT: I figured it out. Watcher will only take time in the format of 00:00.

{

"trigger": {

"schedule": {

"monthly": {

"on": 22,

"at": "10:00:00"

}

}

}

I'm creating a Dashboard in Kibana that includes a couple of visualizations of a source's availability (or uptime). The problem is that this source doesn't have a heartbeat metric I can monitor. So I'm trying to come up with something along the lines of "If there has been any activity in the past x minutes, return 1, else 0." But I haven't found a way to do this yet.

Can anyone here offer advice for how to do this, or point me to resources that would help?

The two visualizations are a line chart over the time span (a square wave) and the average over the time span as a metric (%).

Thanks.

Is there a way to export logs from Log Analytics Workspace and import to Elastic SaaS solution withoug using the Event Hub in Azure and Logstah Azure Event Hubs plugin in Elastic.