Skip to content

Sync the head of k-edge-connect with the main file #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 56 commits into
base: refactor-k-edge-connect
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
9ebdb69
progress on better query algorithm. Not working yet
etwest Nov 10, 2023
23c94ff
working but somewhat slow
etwest Nov 11, 2023
3aaaa64
improved query performance
etwest Nov 14, 2023
ab2c6ec
remove sort bottleneck. Begin investigating post_processing as bottle…
etwest Nov 17, 2023
f1ae69f
make seed a mandatory argument of the CC algorithm
etwest Nov 26, 2023
c0c72d9
merge with refactor and more efficient return type
etwest Nov 29, 2023
ec061c7
oops forgot some files
etwest Nov 29, 2023
c23514c
add spanning forest return type
etwest Nov 29, 2023
3fbf54e
fewer rounds from math
etwest Dec 1, 2023
93d153e
fix tests
etwest Dec 2, 2023
d188fc3
Merge pull request #137 from GraphStreamingProject/refactor_query
etwest Dec 15, 2023
75eb61c
Merge pull request #135 from GraphStreamingProject/refactor
etwest Dec 18, 2023
8546f1d
statistical testing
DanielDeLayo Jan 22, 2024
395089d
Failed sketch merge should leave sketch in bad state
etwest Feb 5, 2024
0ae9902
Merge pull request #141 from GraphStreamingProject/sf-extract
etwest Feb 5, 2024
36c8ff4
adjust to begin incorporation with streaming utilities
etwest Feb 8, 2024
3d0c643
fix the bug and remove more streaming stuff from this repo
etwest Feb 11, 2024
768b4ca
Merge branch 'main' into use_stream_repo
etwest Feb 11, 2024
2534e21
swap unnecessary unique_lock for lock_guard
etwest Feb 12, 2024
057f91b
move to tools
DanielDeLayo Feb 12, 2024
899ba07
Merge branch 'main' into refactor
DanielDeLayo Feb 12, 2024
d2bd087
Merge pull request #142 from GraphStreamingProject/use_stream_repo
etwest Feb 13, 2024
f2ff0f9
Merge branch 'main' into refactor
DanielDeLayo Feb 15, 2024
921dcb2
Merge branch 'refactor' of github.com:GraphStreamingProject/GraphZepp…
DanielDeLayo Feb 15, 2024
d669088
move to tools 2
DanielDeLayo Feb 15, 2024
c88ad19
Merge branch 'main' into refactor
DanielDeLayo Feb 15, 2024
c607b58
Make documentation accurate again
etwest Feb 15, 2024
bdfed7f
GraphSketchDriver doc in README
etwest Feb 16, 2024
085e5ed
get rid of doxy and add initial control flow documentation
etwest Feb 16, 2024
bcdb4a4
Update control_flow.md
etwest Feb 16, 2024
3b3bef4
Update control_flow.md
etwest Feb 16, 2024
f4ccce6
Add description of driver and steps.
etwest Feb 20, 2024
5c45a76
make sketches_factor actually do something and fix exception race-case
etwest Feb 21, 2024
6567d44
also fix log2 constexpr issue
etwest Feb 21, 2024
9795fb0
turn off annoying sketches factor warning
etwest Feb 21, 2024
f7ef3d4
no actually we need to be more thorough
etwest Feb 21, 2024
d502ae4
documentation
DanielDeLayo Feb 22, 2024
86d01d9
Merge branch 'main' into refactor
DanielDeLayo Feb 23, 2024
91106f6
fix bug in pre_insert and optimize dsu find_root
etwest Feb 24, 2024
3d8cf14
add benchmark for find_root
etwest Feb 24, 2024
a3aac1d
remove sketches_factor from process_stream
etwest Feb 24, 2024
736be6d
Update control_flow.md
etwest Feb 25, 2024
b4aa8d5
Merge pull request #139 from GraphStreamingProject/refactor
DanielDeLayo Feb 26, 2024
bfaba5e
Merge branch 'main' into control_sketches
etwest Feb 26, 2024
db06e66
Merge pull request #145 from GraphStreamingProject/control_sketches
etwest Feb 27, 2024
3f00234
better verifier, minimum number of columns, tests
etwest Mar 5, 2024
a69cbf0
remove other verifier files
etwest Mar 5, 2024
f15c7ec
use driver verifier in tests and verify the verifier. Also add query …
etwest Mar 5, 2024
d0f3e5b
Merge pull request #146 from GraphStreamingProject/better_verifier
etwest Mar 21, 2024
bd7d04c
Merge branch 'main' into docs
etwest Jul 24, 2024
f68d883
Merge pull request #143 from GraphStreamingProject/docs
etwest Jul 24, 2024
fd3c971
allow multiple queries in process stream tool
etwest Apr 8, 2025
984ad11
switch batch factor back to 1
etwest Apr 8, 2025
96b53ff
make github workflow work good
etwest May 4, 2025
3813539
Merge branch 'main' into better_tool
etwest May 4, 2025
38a2c3a
Merge pull request #151 from GraphStreamingProject/better_tool
etwest May 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/cmake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, ubuntu-20.04]
os: [ubuntu-latest]
flags: ['"-DL0_SAMPLING"', '"-DNO_EAGER_DSU"', '""']

steps:
Expand Down
8 changes: 1 addition & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,16 +48,10 @@
# other IDEs
/.vscode

#Doxygen docs
/docs/

# Our configuration file
streaming.conf
test/statistical_testing/stat_config.txt

# Mac Files
*.DS_Store

# Python stuff for statistical testing
*__pycache__/
*test_env/

78 changes: 22 additions & 56 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -43,19 +43,19 @@ else()
message (STATUS "GraphZeppelin building executables")
endif()

# Get xxHash
# Get GutterTree Project
FetchContent_Declare(
xxhash
GutterTree

GIT_REPOSITORY https://github.com/Cyan4973/xxHash.git
GIT_TAG v0.8.0
GIT_REPOSITORY https://github.com/GraphStreamingProject/GutterTree.git
GIT_TAG main
)

# Get GutterTree Project
# Get StreamingUtilities
FetchContent_Declare(
GutterTree
StreamingUtilities

GIT_REPOSITORY https://github.com/GraphStreamingProject/GutterTree.git
GIT_REPOSITORY https://github.com/GraphStreamingProject/StreamingUtilities.git
GIT_TAG main
)

Expand All @@ -72,21 +72,7 @@ if (BUILD_BENCH)
FetchContent_MakeAvailable(benchmark)
endif()

FetchContent_MakeAvailable(xxHash GutterTree)
#####
# Some additional steps for xxHash as it is unofficial
#####
#xxHash messes with BUILD_SHARED_LIBS if it is empty
set(SAVED_BUILD_SHARED_LIBS "${BUILD_SHARED_LIBS}")

add_subdirectory(
"${xxhash_SOURCE_DIR}/cmake_unofficial"
"${xxhash_BINARY_DIR}"
EXCLUDE_FROM_ALL
)
#Restore BUILD_SHARED_LIBS
set(BUILD_SHARED_LIBS "${SAVED_BUILD_SHARED_LIBS}" CACHE BOOL "" FORCE)

FetchContent_MakeAvailable(GutterTree StreamingUtilities)

# AVAILABLE COMPILATION DEFINITIONS:
# VERIFY_SAMPLES_F Use a deterministic connected-components
Expand All @@ -102,27 +88,28 @@ set(BUILD_SHARED_LIBS "${SAVED_BUILD_SHARED_LIBS}" CACHE BOOL "" FORCE)

add_library(GraphZeppelin
src/cc_sketch_alg.cpp
src/return_types.cpp
src/driver_configuration.cpp
src/cc_alg_configuration.cpp
src/sketch.cpp
src/util.cpp)
add_dependencies(GraphZeppelin GutterTree)
target_link_libraries(GraphZeppelin PUBLIC xxhash GutterTree)
add_dependencies(GraphZeppelin GutterTree StreamingUtilities)
target_link_libraries(GraphZeppelin PUBLIC xxhash GutterTree StreamingUtilities)
target_include_directories(GraphZeppelin PUBLIC include/)
target_compile_options(GraphZeppelin PUBLIC -fopenmp)
target_link_options(GraphZeppelin PUBLIC -fopenmp)
target_compile_definitions(GraphZeppelin PUBLIC XXH_INLINE_ALL)

add_library(GraphZeppelinVerifyCC
src/cc_sketch_alg.cpp
src/return_types.cpp
src/driver_configuration.cpp
src/cc_alg_configuration.cpp
src/sketch.cpp
src/util.cpp
test/util/file_graph_verifier.cpp
test/util/mat_graph_verifier.cpp)
add_dependencies(GraphZeppelinVerifyCC GutterTree)
target_link_libraries(GraphZeppelinVerifyCC PUBLIC xxhash GutterTree)
test/util/graph_verifier.cpp)
add_dependencies(GraphZeppelinVerifyCC GutterTree StreamingUtilities)
target_link_libraries(GraphZeppelinVerifyCC PUBLIC xxhash GutterTree StreamingUtilities)
target_include_directories(GraphZeppelinVerifyCC PUBLIC include/ include/test/)
target_compile_options(GraphZeppelinVerifyCC PUBLIC -fopenmp)
target_link_options(GraphZeppelinVerifyCC PUBLIC -fopenmp)
Expand All @@ -131,35 +118,19 @@ target_compile_definitions(GraphZeppelinVerifyCC PUBLIC XXH_INLINE_ALL VERIFY_SA
if (BUILD_EXE)
add_executable(tests
test/test_runner.cpp
test/graph_test.cpp
test/cc_alg_test.cpp
test/sketch_test.cpp
test/dsu_test.cpp
test/util_test.cpp
test/util/file_graph_verifier.cpp
test/util/graph_gen.cpp
test/util/graph_gen_test.cpp
test/util/graph_verifier_test.cpp)
add_dependencies(tests GraphZeppelinVerifyCC)
target_link_libraries(tests PRIVATE GraphZeppelinVerifyCC)

add_executable(statistical_test
tools/statistical_testing/graph_testing.cpp
test/util/file_graph_verifier.cpp
test/util/graph_gen.cpp)
add_dependencies(statistical_test GraphZeppelinVerifyCC)
target_link_libraries(statistical_test PRIVATE GraphZeppelinVerifyCC)

# executables for experiment/benchmarking
add_executable(efficient_gen
src/util.cpp
test/util/efficient_gen/edge_gen.cpp
test/util/efficient_gen/efficient_gen.cpp)
target_link_libraries(efficient_gen PRIVATE xxhash GraphZeppelinCommon)

# executable for converting to stream format
add_executable(to_binary_format
tools/to_binary_format.cpp)
target_link_libraries(to_binary_format PRIVATE GraphZeppelinCommon)
add_executable(statistical_sketch_test
tools/sketch_testing.cpp)
add_dependencies(statistical_sketch_test GraphZeppelinVerifyCC)
target_link_libraries(statistical_sketch_test PRIVATE GraphZeppelinVerifyCC)


# executable for processing a binary graph stream
add_executable(process_stream
Expand All @@ -170,12 +141,6 @@ if (BUILD_EXE)
add_executable(test_correctness
tools/test_correctness.cpp)
target_link_libraries(test_correctness PRIVATE GraphZeppelinVerifyCC)

# tool for validating that a binary stream appears correct
add_executable(validate_binary_stream
tools/validate_binary_stream.cpp
)
target_link_libraries(validate_binary_stream PRIVATE GraphZeppelin)
endif()

if (BUILD_BENCH)
Expand All @@ -184,3 +149,4 @@ if (BUILD_BENCH)
add_dependencies(bench_cc GraphZeppelin benchmark)
target_link_libraries(bench_cc GraphZeppelin benchmark::benchmark xxhash)
endif()

Loading