Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crossplane is slow for big configurations #97

Open
BobocIonutGabriel opened this issue Sep 29, 2021 · 0 comments
Open

Crossplane is slow for big configurations #97

BobocIonutGabriel opened this issue Sep 29, 2021 · 0 comments

Comments

@BobocIonutGabriel
Copy link

BobocIonutGabriel commented Sep 29, 2021

When creating(didn't checked loading yet) a big nginx configuration crossplane is slow
Having a big nginx configuration with lots of files creating large ones can be very slow. Sometimes taking more than 30 seconds to create 4 files.

Cause
After some profiling I discovered that what takes the longest is deciding if a string requires quoting. While this is a nice feature, to automatically quote it can get in the way if you know your configuration doesn't require that and you want it to run fast.

Solution
The following patch adds a new parameter that toggles the quoting part if you know you don't need it. This makes the creation of the files almost 10x faster.

diff --git a/crossplane/builder.py b/crossplane/builder.py
index 049d224..8ffdc5e 100644
--- a/crossplane/builder.py
+++ b/crossplane/builder.py
@@ -50,59 +50,59 @@ def _needs_quotes(string):
 
     return char in ('\\', '$') or expanding
 
 
 def _replace_escape_sequences(match):
     return match.group(1).decode('string-escape')
 
 
-def _enquote(arg):
-    if not _needs_quotes(arg):
+def _enquote(arg, no_quote=False):
+    if no_quote or not _needs_quotes(arg):
         return arg
 
     if PY2:
         arg = codecs.encode(arg, 'utf-8') if isinstance(arg, unicode) else arg
         arg = codecs.decode(arg, 'raw-unicode-escape')
         arg = repr(arg).replace('\\\\', '\\').lstrip('u')
         arg = ESCAPE_SEQUENCES_RE.sub(_replace_escape_sequences, arg)
         arg = unicode(arg, 'utf-8')
     else:
         arg = repr(arg).replace('\\\\', '\\')
 
     return arg
 
 
-def build(payload, indent=4, tabs=False, header=False):
+def build(payload, indent=4, tabs=False, header=False, no_quote=False):
     padding = '\t' if tabs else ' ' * indent
 
     head = ''
     if header:
         head += '# This config was built from JSON using NGINX crossplane.\n'
         head += '# If you encounter any bugs please report them here:\n'
         head += '# https://github.com/nginxinc/crossplane/issues\n'
         head += '\n'
 
     def _build_block(output, block, depth, last_line):
         margin = padding * depth
 
         for stmt in block:
-            directive = _enquote(stmt['directive'])
+            directive = _enquote(stmt['directive'], no_quote)
             line = stmt.get('line', 0)
 
             if directive == '#' and line == last_line:
                 output += ' #' + stmt['comment']
                 continue
             elif directive == '#':
                 built = '#' + stmt['comment']
             elif directive in EXTERNAL_BUILDERS:
                 external_builder = EXTERNAL_BUILDERS[directive]
                 built = external_builder(stmt, padding, indent, tabs)
             else:
-                args = [_enquote(arg) for arg in stmt['args']]
+                args = [_enquote(arg, no_quote) for arg in stmt['args']]
 
                 if directive == 'if':
                     built = 'if (' + ' '.join(args) + ')'
                 elif args:
                     built = directive + ' ' + ' '.join(args)
                 else:
                     built = directive
 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant