Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integrate with alloy-analyzer #13

Merged
merged 30 commits into from
Jul 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
cc77951
use tagged action
tcm5343 Mar 8, 2024
5b16941
update pr verifcation
tcm5343 Mar 8, 2024
e27f613
add logging in container
tcm5343 Mar 8, 2024
60ee155
print out testing data
tcm5343 Mar 8, 2024
22d6687
fix paths
tcm5343 Mar 8, 2024
bd35868
use context iinstead
tcm5343 Mar 8, 2024
db3d081
hardcode it
tcm5343 Mar 8, 2024
5e6531e
fix local too
tcm5343 Mar 8, 2024
6e2db4f
make the input graph first
tcm5343 Mar 8, 2024
ab8f7fe
update usage
tcm5343 Mar 8, 2024
2392bbc
link to todo in readme
tcm5343 Mar 9, 2024
139e920
init go talking to java and streaming the file
tcm5343 Jun 27, 2024
7dedb25
add gitattributes
tcm5343 Jun 27, 2024
302ba9e
write to and read from alloy analyzer and use input in unit tets
tcm5343 Jul 21, 2024
1efbbdc
fix lint issue
tcm5343 Jul 21, 2024
da13c00
fix branch name
tcm5343 Jul 21, 2024
454cfe0
fix order
tcm5343 Jul 21, 2024
baf06cc
fix order and pathing now
tcm5343 Jul 21, 2024
a6dda28
fix order and pathing now
tcm5343 Jul 21, 2024
a12547c
update code owners
tcm5343 Jul 22, 2024
936887e
run unit tests
tcm5343 Jul 22, 2024
9b1a80e
just checkout files
tcm5343 Jul 22, 2024
df7edf5
fix path
tcm5343 Jul 22, 2024
7af99b7
start analyzer service
tcm5343 Jul 22, 2024
d9100cc
add dot package
tcm5343 Jul 26, 2024
cf2be62
add changes
tcm5343 Jul 26, 2024
cb1b48b
add rpc pacakge and simplify testing code
tcm5343 Jul 26, 2024
e1ec2b4
clean up some comments
tcm5343 Jul 26, 2024
163f2ac
clean up workflow
tcm5343 Jul 27, 2024
2f7524c
Merge branch 'main' into dev
tcm5343 Jul 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .env-example

This file was deleted.

1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.sh text eol=lf
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* @tcm5343 @rory-tatum @pabs159
* @tcm5343
30 changes: 19 additions & 11 deletions .github/workflows/pull_request_validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,37 @@ on:
- main

jobs:
cdd-e2e:
unit-test:
runs-on: ubuntu-latest
# strategy:
# matrix:
# input_file:
# - adj_list_cycle.txt
# - adj_list_no_cycle.txt
steps:
- name: checkout current branch
uses: actions/checkout@v4
with:
repository: ${{ github.repository }}
path: ${{ github.repository }}
ref: ${{ github.GITHUB_REF_NAME }} # current branch name

- name: checkout alloy-analyzer-service
uses: actions/checkout@v4
with:
repository: tcm5343/alloy-analyzer-service
path: alloy-analyzer-service

- name: install task
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: create graph input file

- name: start alloy-analyzer-service
run: |
echo -e "1 2\n2 3\n3 1" > testing/data/some-input-graph.txt
cd alloy-analyzer-service
docker compose up --build --detach
cd ..

- name: task run
- name: run unit tests
run: |
task INPUT_FILE="testing/data/some-input-graph.txt" run
ls
pwd
cd ${{ github.repository }}
task test
cd ..
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,6 @@ go.work

.vscode/
.task/
/tmp

*.env
41 changes: 19 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,51 +8,48 @@ This project was first implemented at the University of Texas at Austin as a gro

## Usage

Below is an example GitHub Action job which utilizes the circular dependency detector. Simply write you graph input file to `/testing/data` and then pass that into the detector.
Below is an example GitHub Action job which utilizes the circular dependency detector. Simply write you graph input file to `/testing/data` and then pass that into the detector. Reminder that the checkout location in an Action is `/github/workspace`. Only Ubuntu runners support containerized Actions.

```yaml
jobs:
cdd:
runs-on: ubuntu-latest
steps:
- name: checkout cdd
uses: actions/checkout@v4
with:
repository: ${{ github.repository }}
ref: ${{ github.GITHUB_REF_NAME }} # this should be a working tag

- name: install task
uses: arduino/setup-task@v2
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: create graph input file
run: |
echo -e "1 2\n2 3\n3 1" > testing/data/some-input-graph.txt
echo -e "1 2\n2 3\n3 1" > some-input-graph.txt

- name: circular dependency detector
run: |
task INPUT_FILE="testing/data/some-input-graph.txt" run
- name: circular-dependency-detector
uses: tcm5343/circular-dependency-detector@<TODO LATEST VERSION HERE>
with:
adjacency_list_path: /github/workspace/some-input-graph.txt
```


## Contributing

Command for generating the proto files: protoc --go_out=. --go-grpc_out=. --go_opt=paths=source_relative --go-grpc_opt=paths=source_relative protos/filestream.proto


### Software

* [Task](https://taskfile.dev/) shall be used in the CI/CD pipelines and for local development for orchestration. Run `task` for the list of tasks.
* [Podman](https://podman.io/) is used to manage containers in the`Taskfile`.
* [golangci-lint](https://golangci-lint.run/) is used for lint.

### Configuration
### Local Configuration

For local development, create a `.env` file at the root of the repository to modify your config. The only supported format for the input graph file environment variable (`INPUT_FILE`) is an adjacency list which follows the format used by [NetworkX](https://networkx.org/documentation/stable/reference/readwrite/adjlist.html#). An `.env-example` file exists at the root of the project.
For local development, create a `.env` file at the root of the repository to use as your config. The only supported format for the input graph file environment variable (`INPUT_FILE`) is an adjacency list which follows the format used by [NetworkX](https://networkx.org/documentation/stable/reference/readwrite/adjlist.html#). An important note for debugging is the values in `.env` are simply passed to the app through positional arguments.

```shell
user@machine:~/dev/circular-dependency-detector$ cat ./.env
INPUT_FILE=testing/data/adj_list_no_cycle.txt # no spaces for now, defaults to {i don't know yet}
```Dotenv
INPUT_FILE=/github/workspace/testing/data/adj_list_no_cycle.txt # no spaces for now, defaults to {i don't know yet}
FAIL_ON_CYCLE=true # default is false
OUTPUT_FILE="/github/workspace/"
```

### Roadmap

Future work is identified in [TODO.md](TODO.md).

## Musical Acknowledgements

Bob Dylan - Early Mornin' Rain</br>
2 changes: 0 additions & 2 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
## Todo

* In a workflow, create a input graph file, and execute the action with it
* Determine if project structure is idiomatic
* Determine if `.env` or CLI args are better for configuration
* Build out CI/CD pipeline:
* Add end to end testing of the program
* Add linting to the pipeline
Expand Down
11 changes: 6 additions & 5 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ tasks:
build:
desc: build the container
cmds:
- podman build -t circular-dependency-detector .
- docker build -t circular-dependency-detector .
sources:
- ./**/*.go
- ./**/*.mod
Expand All @@ -20,26 +20,27 @@ tasks:
- ./entrypoint.sh
- testing/**/*
- Dockerfile
- Taskfile.yml

run:
desc: run the program locally inside container
deps: [build]
cmds:
- podman run circular-dependency-detector "{{ .INPUT_FILE }}"
- docker run circular-dependency-detector "{{ .INPUT_FILE }}" "{{ .FAIL_ON_CYCLE }}"

test:
desc: run the unit tests
deps: [build]
cmds:
- podman run --entrypoint 'go' circular-dependency-detector test -v /app/...
- docker run --entrypoint 'go' circular-dependency-detector test -v /app/...

lint:
desc: runs lint validation
cmds:
- golangci-lint run
sources:
- './**/*.go'
- .golangci.yml
cmds:
- golangci-lint run

# clean:
# cmds:
Expand Down
48 changes: 14 additions & 34 deletions action.yml
Original file line number Diff line number Diff line change
@@ -1,41 +1,21 @@
name: 'circular-dependency-detector'
description: 'Detect and output cycles and topological generations given a directed graph.'
inputs:
adjacency_list_path:
description: 'Path to a file where an adjacency list is stored'
INPUT_PATH:
description: 'absolute path to graph input file'
required: true
runs:
FAIL_ON_CYCLE:
description: 'flag if action should fail if a cycle is detected'
required: false
default: true
OUTPUT_FILE:
description: 'absolute path to graph output file'
required: false
default: '/github/workspace'
using: "docker"
image: 'Dockerfile'
# the order of these args matter to the internal running of the app
args:
- ${{ inputs.adjacency_list_path }}

# name: 'circular-dependency-detector'
# description: 'Detect and output cycles and topological generations given a directed graph.'
# inputs:
# adjacency_list_path:
# description: 'Path to a file where an adjacency list is stored'
# required: true
# runs:
# using: "composite"
# steps:
# - name: Download a single artifact
# uses: actions/download-artifact@v3
# with:
# name: adj-list

# - name: Print
# shell: bash
# run: |
# pwd
# ls

# - name: Build Container
# shell: bash
# run: |
# docker build -t circular-dependency-detector --progress=plain --no-cache --build-arg ADJACENCY_LIST_PATH=${{ inputs.adjacency_list_path }} .

# - name: Run Container
# shell: bash
# run: |
# docker run circular-dependency-detector
- ${{ inputs.INPUT_PATH }}
- ${{ inputs.FAIL_ON_CYCLE }}
- ${{ inputs.OUTPUT_FILE }}
Binary file added cmd/cmd
Binary file not shown.
38 changes: 23 additions & 15 deletions cmd/main.go
Original file line number Diff line number Diff line change
@@ -1,49 +1,57 @@
package main

import (
"flag"
"log"
"log/slog"
"os"

"gonum.org/v1/gonum/graph/topo"

"github.com/tcm5343/circular-dependency-detector/pkg/graph"
internalGraph "github.com/tcm5343/circular-dependency-detector/pkg/graph"
)

func main() {
argsWithoutProg := os.Args[1:]
// todo: maybe print out the args
inputGraphPath := flag.String("input-file", "SOME DEFAULT PATH", "absolute path to graph input file")
failOnCycle := flag.Bool("fail-on-cycle", true, "flag if action should fail if a cycle is detected")
flag.Parse()

slog.SetLogLoggerLevel(slog.LevelInfo)
slog.SetLogLoggerLevel(slog.LevelDebug)
log.SetFlags(log.LstdFlags | log.Lshortfile)

// fmt.Println(argsWithoutProg)
// fmt.Println("/app/" + inputGraphPath)
var inputGraphPath string
val, present := os.LookupEnv("INPUT_FILE")
if present {
inputGraphPath = val
} else {
inputGraphPath = "/app/" + argsWithoutProg[0]
}
// var inputGraphPath string
// inputFile, present := os.LookupEnv("INPUT_FILE")
// if present {
// inputGraphPath = inputFile
// } else {
// inputGraphPath = argsWithoutProg[0]
// }

fp, err := os.Open(inputGraphPath) // fix: this path can't contain spaces for some reason...
fp, err := os.Open(*inputGraphPath) // fixme: this path can't contain spaces for some reason...
if err != nil {
panic(err)
}
defer fp.Close()

lg, err := graph.ParseInputGraph(fp)
lg, err := internalGraph.ParseInputGraph(fp)
if err != nil {
panic(err)
}

cycles := topo.DirectedCyclesIn(lg.Graph)
// cycles := topo.DirectedCyclesIn(g)
slog.Debug("directed cycles in", "count", len(cycles), "cycles", cycles)

if len(cycles) > 0 && *failOnCycle {
slog.Error("failing due to cycles being found")
os.Exit(1)
}

if len(cycles) > 0 {
slog.Info("skipping topological generations . . .")
} else {
tg, err := graph.TopologicalGenerationsOf(lg.Graph)
tg, err := internalGraph.TopologicalGenerationsOf(lg.Graph)
if err != nil {
panic(err)
}
Expand Down
6 changes: 2 additions & 4 deletions entrypoint.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
#!/bin/sh
# !/bin/sh

set -e

INPUT_FILE=$1

/app/circular-dependency-detector $INPUT_FILE
/app/circular-dependency-detector --input-file="$1" --fail-on-cycle="$2"
# python3 ./visualizer/visualize.py
13 changes: 12 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,15 @@ module github.com/tcm5343/circular-dependency-detector

go 1.22

require gonum.org/v1/gonum v0.15.0
require (
gonum.org/v1/gonum v0.15.0
google.golang.org/grpc v1.64.0
google.golang.org/protobuf v1.34.2
)

require (
golang.org/x/net v0.22.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/text v0.16.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237 // indirect
)
14 changes: 14 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
golang.org/x/net v0.22.0 h1:9sGLhx7iRIHEiX0oAJ3MRZMUCElJgy7Br1nO+AMN3Tc=
golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.16.0 h1:a94ExnEXNtEwYLGJSIUxnWoxoRz/ZcCsV63ROupILh4=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237 h1:NnYq6UN9ReLM9/Y01KWNOWyI5xQ9kbIms5GGJVwS/Yc=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237/go.mod h1:WtryC6hu0hhx87FDGxWCDptyssuo68sk10vYjF+T9fY=
google.golang.org/grpc v1.64.0 h1:KH3VH9y/MgNQg1dE7b3XfVK0GsPSIzJwdF617gUSbvY=
google.golang.org/grpc v1.64.0/go.mod h1:oxjF8E3FBnjp+/gVFYdWacaLDx9na1aqy9oovLpxQYg=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa h1:FRnLl4eNAQl8hwxVVC17teOw8kdjVDVAiFMtgUdTSRQ=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa/go.mod h1:zk2irFbV9DP96SEBUUAy67IdHUaZuSnrz1n472HUCLE=
gonum.org/v1/gonum v0.15.0 h1:2lYxjRbTYyxkJxlhC+LvJIx3SsANPdRybu1tGj9/OrQ=
Expand Down
Loading