Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import less specific CIDR if abuse-c is missing #8

Open
bernhardreiter opened this issue Jun 6, 2018 · 1 comment
Open

Import less specific CIDR if abuse-c is missing #8

bernhardreiter opened this issue Jun 6, 2018 · 1 comment

Comments

@bernhardreiter
Copy link
Member

Problem: Some CIDRs with a specific country code do not have an abuse email address via
abuse-c or the org, but a wider CIDR that covers the range has and is from a different country.

Right now the wider CIDR is not imported because it is from a different country.
So we do not find an abuse contact for the CIDR.

Technical solution idea

If no abuse mail-address is found in the CIDR for a country, look into wider CIDRs covering the range
until one with an abuse address is found. The first one that is found, will be imported.

@bernhardreiter
Copy link
Member Author

bernhardreiter commented Jun 6, 2018

Implementation considerations

  • Right now filtering by country is done early, e.g. in https://github.com/Intevation/intelmq/blob/certbund-contact/intelmq/bots/experts/certbund_contact/ripe_data.py#L112 to reduce the amount of data early to speed up processing. This would need to be changed because we will need the information for all countries to search for less specific CIDRs.
  • Next we need an efficient way to find out the next wider CIDR, first idea is to build a tree data structure so that more specific CIDRs are below the wider one and we can go up the tree. This would only be the effort of building the tree, otherwise we need to run over all CIDRs to find out which is the next wider one. Needs research because probably index data structures for this already exist, probably in python modules.
  • After finding the INETNUM and INET6NUM we want to import additionally to the one found by country, we need to remove the other ones.
  • Code after this and diff should be fine, but we need to test this at least with a few samples.
  • There are speed reasons to consider, we probably have to create a small test set of ripe_files to be able to do fast developement runs.

@bernhardreiter bernhardreiter transferred this issue from Intevation/intelmq Sep 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant