Add check-es-index-field-count.rb #112
Conversation
|
Tested locally.
|
bin/check-es-indices-sizes.rb
Outdated
| @@ -161,9 +161,9 @@ def run | |||
| nodes_being_used = node_fs_stats['nodes'].values.select { |node| node['indices']['store']['size_in_bytes'] > 0 } | |||
| # TODO: come back and cleanup all these rubocop disables with a little refactor | |||
| # rubocop:disable Style/SingleLineBlockParams | |||
| used_in_bytes = nodes_being_used.map { |node| node['fs']['data'].map { |data| data['total_in_bytes'] - data['available_in_bytes'] }.flatten }.flatten.inject { |sum, x| sum + x } # rubocop:disable LineLength | |||
| # rubocop:disable LineLength | |||
There was a problem hiding this comment.
if you are not disabling rubocop inline you should then enable it when you are done.
There was a problem hiding this comment.
rubocop is complaining about line being too long here, hence the disable directive.
There was a problem hiding this comment.
# rubocop:disable Metrics/LineLength on the same line.
bin/check-es-index-field-count.rb
Outdated
| class ESIndexFieldCount < Sensu::Plugin::Check::CLI | ||
| include ElasticsearchCommon | ||
|
|
||
| option :transport, |
There was a problem hiding this comment.
I don't see this being used anywhere do we need this? If support is not implemented then we should not include it.
bin/check-es-index-field-count.rb
Outdated
| long: '--transport TRANSPORT', | ||
| description: 'Transport to use to communicate with ES. Use "AWS" for signed AWS transports.' | ||
|
|
||
| option :region, |
There was a problem hiding this comment.
I don't see this being used anywhere do we need this? If support is not implemented then we should not include it.
|
For bonus points you could write an integration test, we currently spin up an ES6 container in travis and you could add creating an index with field mappings and use that to write a test against. |
|
@majormoses Integration test is added |
bin/check-es-index-field-count.rb
Outdated
| # | ||
| # DEPENDENCIES: | ||
| # gem: sensu-plugin | ||
| # gem: aws_es_transport |
There was a problem hiding this comment.
As this is clearly not being maintained anymore I think we should not include it, we are gonna rip that out from the checks in the somewhat near future.
cdunn/aws-es-transport#3
bin/check-es-index-field-count.rb
Outdated
| Use `_all` or empty string to perform the operation on all indices. | ||
| Accepts wildcards', | ||
| short: '-i INDEX', | ||
| long: '--indices INDEX' |
There was a problem hiding this comment.
I think we should default to _all as I imagine most people want to check all indices.
bin/check-es-index-field-count.rb
Outdated
| end | ||
|
|
||
| def run | ||
| config[:index] = '_all' if config[:index].nil? |
There was a problem hiding this comment.
why not just default to it rather than embedding this logic?
|
Can you add the new check to the list of files in the readme? |
|
@majormoses Comments are addressed |
bin/check-es-indices-sizes.rb
Outdated
| @@ -161,9 +161,9 @@ def run | |||
| nodes_being_used = node_fs_stats['nodes'].values.select { |node| node['indices']['store']['size_in_bytes'] > 0 } | |||
| # TODO: come back and cleanup all these rubocop disables with a little refactor | |||
| # rubocop:disable Style/SingleLineBlockParams | |||
| used_in_bytes = nodes_being_used.map { |node| node['fs']['data'].map { |data| data['total_in_bytes'] - data['available_in_bytes'] }.flatten }.flatten.inject { |sum, x| sum + x } # rubocop:disable LineLength | |||
| # rubocop:disable LineLength | |||
There was a problem hiding this comment.
# rubocop:disable Metrics/LineLength on the same line.
Clean up rubocop warnings Add test for check-es-index-field-count Add check-es-index-field-count to README
Pull Request Checklist
#104
General
Update Changelog following the conventions laid out here
Update README with any necessary configuration snippets
Binstubs are created if needed
RuboCop passes
Existing tests pass
New Plugins
Tests
Add the plugin to the README
Does it have a complete header as outlined here
Purpose
This is a check to make sure we get alerts when the number of fields is approaching the targeted limit. Elasticsearch by defaults sets this limit to 1000. While there are certainly use cases when you may even need more than this, but in general it's an indicator your index is being too "flat". Hence you may want to review your mapping strategy. This check helps us know beforehand when the limit is being about to be reached so that you can have the proper action, for example, use another index or increase index.mapping.total_fields.limit setting.
Known Compatibility Issues