Using ANT patterns when uploading

[mariakoleva]

We have a use case to upload parts of a directory tree to Artifactory by excluding and including files and directories using ANT patterns. We need to provide a way to our users to specify multiple ANT patterns to be excluded from their upload and multiple ANT patterns to be the only files/directories included in their upload.

I'd like to see an option on the upload JFrog CLI command similar to the --exclusions option but supporting ANT patterns instead of regular expressions. I'd also like to see a similar --inclusions option that works with ANT patterns.

At the moment we are doing pattern matching ourselves, before creating a compressed archive of the directory tree and passing that to the JFrog CLI upload command. However, we are seeing bad performance with this implementation and think this can be sped up if it was implemented as part of the JFrog CLI command.

This is related to issue #867 - that is the other feature request needed for the same use case.

[eyalbe4]

@mariakoleva,

Thanks for sharing this requirement!
Have you tried using the jfrog rt u command without regular expressions, meaning without the --regexp option? If you don't use this option, the command expects patterns which are very similar to ANT patterns. I believe that if you send an ANT pattern to the command, it will work as expected. This is because the double asterisk syntax of ANT patterns works seamlessly with the wildcards patent accepted by JFrpg CLI.

Would you like to try it out and let us know?

[mariakoleva]

Hi Eyal, Thank you for your reply! I had not tried that. It is great to hear this might already be in place. I tried a couple of tests cases for exclusion that we have and noticed the below: When using this exclude pattern “**/dir/**/*.txt” the following doesn’t get excluded: dir/a.txt. As far as I know, ** should match 0 or more directories. I’ve also got a question on the inclusion. Is there a way to pass in multiple include patterns? So far I’ve been able to include by using a pattern in the first argument to the upload command. But I’m not sure how to do multiple include patterns. Thank you, Maria

[eyalbe4]

Hi @mariakoleva,

For passing multiple include patterns for the jfrog rt upload command , I recommend using File Specs. See Example 4: Upload, which uploads files to Artifactory from two different local paths. You can then pass the file spec path to the command using the --spac option.

As for the example you showed, in which you provided the “**/dir/**/*.txt” pattern, this is actually one of the ways ANT patterns differ from plain wildcard patterns. Changing the pattern you provided to either “dir/*.txt” or “*dir/*.txt” will match "dir/a.txt". And if you need to be more specific, you can provide multiple patterns.

Please let me know if this helps.

[mariakoleva]

Hi,

Thank you for your reply. I haven't tried file specs but they look useful for passing in multiple patterns. I suppose this feature request can be amended to a request to support ANT patterns in file specs.

Unfortunately, changing from ANT patterns to plain wildcard patterns is not an option for us. We have a large number of CI jobs using ANT patterns which we are switching over to Artifactory. It's not feasible to change all of them to wildcard patterns.

Thanks,

Maria

[eyalbe4]

@mariakoleva,
The latest release of JFrog CLI adds a new --ant option to the jfrog rt u command.
Looking forward to your feedback for it.

[mariakoleva]

Hi @eyalbe4,

Thank you for adding this! I finally had the chance to test the new --ant option. I tested 25 use cases with it, and only found a few issues. I've raised all of those as support tickets already but I'm happy to share the ticket numbers with you if that will be helpful.

[eyalbe4]

Thank you for the feedback @mariakoleva!
We'll happily add improvements and fixes following the support investigation.

[enaess]

As am trying this, is there support for variable matching too, e.g.

{
  "files": [
    {   
      "pattern": "target/build/**/{file:\\w}*.jar",
      "target": "some/path/{file}.jar,
      "ant": true
    }
}

[shuvadipc]

supporting ANT patterns when uploading is already supported.