[PingDirectory][Http Access] Add Http Access data stream#19467
Open
muskan-agarwal26 wants to merge 2 commits into
Open
[PingDirectory][Http Access] Add Http Access data stream#19467muskan-agarwal26 wants to merge 2 commits into
muskan-agarwal26 wants to merge 2 commits into
Conversation
Contributor
Elastic Docs Style Checker (Vale)Summary: 1 warning, 1 suggestion found
|
| File | Line | Rule | Message |
|---|---|---|---|
| packages/ping_directory/data_stream/http_access/fields/fields.yml | 6 | Elastic.Latinisms | Latin terms and abbreviations are a common source of confusion. Use 'for example' instead of 'e.g'. |
💡 Suggestions (1): Optional style improvements. Apply when helpful.
| File | Line | Rule | Message |
|---|---|---|---|
| packages/ping_directory/changelog.yml | 1 | Elastic.Versions | Use 'later versions' instead of 'newer versions' when referring to versions. |
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.
|
✅ All changelog entries have the correct PR link. |
🚀 Benchmarks reportTo see the full report comment with |
💚 Build Succeeded
|
efd6
reviewed
Jun 10, 2026
Comment on lines
+169
to
+173
| - set: | ||
| field: event.duration | ||
| tag: set_event_duration_from_http_access_etime | ||
| copy_from: ping_directory.http_access.etime | ||
| ignore_empty_value: true |
Contributor
There was a problem hiding this comment.
This needs to be scaled. The API returns etime in milliseconds (ref).
| tag: append_error_message_for_convert_responseContentLength_to_long | ||
| value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' | ||
| - convert: | ||
| field: http.response.status_code |
Contributor
There was a problem hiding this comment.
This field can be used to populate event.outcome.
| field: event.original | ||
| tag: parse_http_access | ||
| patterns: | ||
| - '\[(?<ping_directory.http_access.timestamp>\d{2}\/[A-Za-z]{3}\/\d{4}:\d{2}:\d{2}:\d{2}\.\d{3}) (?<ping_directory.http_access.timezone>[+-]\d{4})\] %{WORD:ping_directory.http_access.type}(?: instanceName="%{DATA:observer.name}")?(?: threadID=%{NUMBER:ping_directory.http_access.thread_id:long})?(?: requestID=%{NUMBER:ping_directory.http_access.request_id})?(?: correlationID="%{DATA:ping_directory.http_access.correlation_id}")?(?: from="%{IP:client.ip}:%{NUMBER:client.port:long}")?(?: method="%{WORD:http.request.method}")?(?: url="%{DATA:url.full}")?(?<_tmp_request_headers>(?: requestHeader="[^"]*")*)(?: authorizationType="%{DATA:ping_directory.http_access.authorization_type}")?(?: requestContentType="%{DATA:http.request.mime_type}")?(?: requestContentLength=%{NUMBER:http.request.bytes:long})?(?: statusCode=%{NUMBER:http.response.status_code:long})?(?: etime=%{NUMBER:ping_directory.http_access.etime:float})?(?: responseContentLength=%{NUMBER:http.response.bytes:long})?(?<_tmp_response_headers>(?: responseHeader="[^"]*")*)(?: responseContentType="%{DATA:http.response.mime_type}")?(?: redirectURI="%{DATA:ping_directory.http_access.redirect_uri}")?' |
Contributor
There was a problem hiding this comment.
- Do a
uri_partson theurl.fullfrom here. Also setnetwork.protocol` from the scheme. - Do a
geoipenrichment ofclient.ip. - There are converts implied here, so the
converts that follow for these fields are not required for the:longand:floatmatch specifiers.
| copy_from: ping_directory.http_access.request_header.referer | ||
| ignore_empty_value: true | ||
| - set: | ||
| field: user_agent.original |
Contributor
There was a problem hiding this comment.
Map this to user_agent.* fields with a user_agent processor.
| ignore_empty_value: true | ||
| - append: | ||
| field: trace.id | ||
| tag: append_ping_directory_http_access_response_header_correlation_id_into_trace_id |
Contributor
There was a problem hiding this comment.
Before, this was set. I'm pretty sure this is normalised to a scalar (there is not "array" expectation here).
Comment on lines
+16
to
+21
| {{#if tz_offset}} | ||
| fields_under_root: true | ||
| fields: | ||
| _conf: | ||
| tz_offset: "{{tz_offset}}" | ||
| {{/if}} |
Contributor
There was a problem hiding this comment.
This is not defined in a manifest.
Comment on lines
+10
to
+16
| - name: paths | ||
| type: text | ||
| title: Paths | ||
| multi: true | ||
| required: true | ||
| show_user: true | ||
| description: A list of glob-based paths that will be crawled and fetched. |
Contributor
There was a problem hiding this comment.
Is there a sensible default for this?
|
|
||
| ### Agent-based deployment | ||
|
|
||
| For more details, refer to the Elastic Agent installation instructions. -> For more details, check the Elastic Agent [installation instructions](docs-content://reference/fleet/install-elastic-agents.md). You can install only one Elastic Agent per host. |
Contributor
There was a problem hiding this comment.
Suggested change
| For more details, refer to the Elastic Agent installation instructions. -> For more details, check the Elastic Agent [installation instructions](docs-content://reference/fleet/install-elastic-agents.md). You can install only one Elastic Agent per host. | |
| For more details, check the Elastic Agent [installation instructions](docs-content://reference/fleet/install-elastic-agents.md). You can install only one Elastic Agent per host. |
|
|
||
| For more details, refer to the Elastic Agent installation instructions. -> For more details, check the Elastic Agent [installation instructions](docs-content://reference/fleet/install-elastic-agents.md). You can install only one Elastic Agent per host. | ||
|
|
||
| Elastic Agent is required to stream data from the syslog or log file receiver and ship the data to Elastic, where the events will then be processed via the integration's ingest pipelines. |
Contributor
There was a problem hiding this comment.
Do all of the visualisation in this dashboard have titles?
andrewkroh
added
New Integration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Checklist
changelog.ymlfile.How to test this PR locally
To test the PingDirectory package:
Related issues
Screenshots