crossplane.parse('nginx.conf') fails with a valid nginx config file.

[Abuelodelanada]

Describe the bug
crossplane.parse('nginx.conf') fails with a valid nginx config file.

To Reproduce
Steps to reproduce the behavior:

1. Take the following valid nginx config file (nginx.conf) with the this content:

worker_processes  1;

events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    upstream self {
      server localhost:80;
    }

    server {
        listen               80;
        server_name          localhost;
        root                 /web;

	location ~ ^/[a-z0-9_-]+$ {
	 rewrite            /.* /;
	 proxy_pass         http://localhost:80;
	 proxy_redirect     off;
	 proxy_set_header   Host $host;
	}

        location  ~ /[a-z0-9_-]+/ {
	 if ($request_uri ~* "/[a-z0-9_-]+/(.*)") { 
            proxy_pass       http://localhost:80/$1;
          }   

          proxy_redirect     off;
          proxy_set_header   Host $host;
        }

	location ~ ^/[a-z0-9_-]+/(.*)$ {
	 rewrite            /[a-z0-9_-]+/(.*) /$2;
  	 proxy_pass         http://localhost:80;
          proxy_redirect     off;
          proxy_set_header   Host $host;
	}

        location / {
            index            index.html index.htm;
        }

        error_page           500 502 503 504  /50x.html;
        location = /50x.html {
            root             /usr/share/nginx/html;
        }
    }
}

2. Run conf = crossplane.parse('nginx.conf') on nginx.conf file
3. See the following errors:

In [40]: conf["status"]
Out[40]: 'failed'

In [41]: conf["errors"]
Out[41]: 
[{'file': 'nginx.conf',
  'error': "[Errno 2] No such file or directory: 'mime.types'",
  'line': 9}]

4. If I try to run .build() with this config I get:

In [6]: crossplane.build(conf["config"])
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[6], line 1
----> 1 crossplane.build(conf["config"])

File ~/trabajos/canonical/env/lib/python3.11/site-packages/crossplane/builder.py:122, in build(payload, indent, tabs, header)
    119     return output
    121 body = ''
--> 122 body = _build_block(body, payload, 0, 0)
    123 return head + body

File ~/trabajos/canonical/env/lib/python3.11/site-packages/crossplane/builder.py:88, in build.<locals>._build_block(output, block, depth, last_line)
     85 margin = padding * depth
     87 for stmt in block:
---> 88     directive = _enquote(stmt['directive'])
     89     line = stmt.get('line', 0)
     91     if directive == '#' and line == last_line:

KeyError: 'directive'

In [7]: 

Expected behavior

I expect that crossplane.parse() parse the config file without any issues since this config is used by a nginx instance without any errors.

Your environment

• Operating System: Debian GNU/Linux 11 (bullseye)
• Version of crossplane: crossplane 0.5.8

[ryepup]

In the example nginx.conf, the include mime.types; directive is saying that part of this config is in another file. If that file is missing, then nginx.conf isn't considered valid, because we're missing information from that mime.types file. It's probably working in your nginx instance because the mime.types file is present.

Many NGINX users split their config into different files, combining them via includes. crossplane.parse attempts to read any includeed files so it can assemble the entire configuration into one object.

I see two options to get the parser happy:

1. put a mime.types file in the same directory as the nginx.conf
2. drop the include mime.types; directive

[Abuelodelanada]

Hi @ryepup !

you were right, the mime.types was not there. Adding that file I got the following:

In [8]: conf = crossplane.parse('nginx.conf')

In [9]: conf["status"]
Out[9]: 'ok'

In [10]: conf["errors"]
Out[10]: []

In [11]: crossplane.build(conf["config"])
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[11], line 1
----> 1 crossplane.build(conf["config"])

File ~/trabajos/canonical/env/lib/python3.11/site-packages/crossplane/builder.py:122, in build(payload, indent, tabs, header)
    119     return output
    121 body = ''
--> 122 body = _build_block(body, payload, 0, 0)
    123 return head + body

File ~/trabajos/canonical/env/lib/python3.11/site-packages/crossplane/builder.py:88, in build.<locals>._build_block(output, block, depth, last_line)
     85 margin = padding * depth
     87 for stmt in block:
---> 88     directive = _enquote(stmt['directive'])
     89     line = stmt.get('line', 0)
     91     if directive == '#' and line == last_line:

KeyError: 'directive'

If I remove the include directive from nginx.conf file I get:

In [16]: conf = crossplane.parse('nginx.conf')

In [17]: conf["status"]
Out[17]: 'ok'

In [18]: conf["errors"]
Out[18]: []

In [19]: crossplane.build(conf["config"])
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[19], line 1
----> 1 crossplane.build(conf["config"])

File ~/trabajos/canonical/env/lib/python3.11/site-packages/crossplane/builder.py:122, in build(payload, indent, tabs, header)
    119     return output
    121 body = ''
--> 122 body = _build_block(body, payload, 0, 0)
    123 return head + body

File ~/trabajos/canonical/env/lib/python3.11/site-packages/crossplane/builder.py:88, in build.<locals>._build_block(output, block, depth, last_line)
     85 margin = padding * depth
     87 for stmt in block:
---> 88     directive = _enquote(stmt['directive'])
     89     line = stmt.get('line', 0)
     91     if directive == '#' and line == last_line:

KeyError: 'directive'

Here are the two files: nginx.conf and mime.types: nginx-config.zip

[ryepup]

This line is the problem:

crossplane.build(conf["config"])

The build function does not accept the whole config dict, it needs just the "parsed" data. The config dict has other metadata that's not relevant for building the nginx config file. You can see an example in the format CLI command, which calls parse and then build:

crossplane/crossplane/formatter.py

Line 7 in 3e616a8

def format(filename, indent=4, tabs=False):