Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get bits #1115

Open
wants to merge 62 commits into
base: development
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
1de6d17
get bits
danielhrisca Dec 9, 2024
acd0fc5
covnert test
danielhrisca Dec 9, 2024
6503bef
workign threads on windows
danielhrisca Dec 9, 2024
8161a80
get parallel
danielhrisca Dec 10, 2024
fa88aa5
print
danielhrisca Dec 10, 2024
e05f8c7
linux threads
danielhrisca Dec 10, 2024
efbd08f
style
danielhrisca Dec 10, 2024
a15dc8a
Merge branch 'development' of https://github.com/danielhrisca/asammdf…
danielhrisca Dec 10, 2024
8581d75
remove code duplication in the extend method
danielhrisca Dec 10, 2024
d993fb1
Merge branch 'development' of https://github.com/danielhrisca/asammdf…
danielhrisca Dec 11, 2024
00f2ab5
parallel data_block_from_arrays
danielhrisca Dec 11, 2024
e4521db
use store_size to optimize the decompression
danielhrisca Dec 11, 2024
bf23b48
small fixes
danielhrisca Dec 11, 2024
3154690
Timer class
danielhrisca Dec 11, 2024
c91348a
truely parallel get raw bytes
danielhrisca Dec 11, 2024
c365316
start rework on the get_invalidation_bits
danielhrisca Dec 11, 2024
adbc444
fix
danielhrisca Dec 12, 2024
94618b2
void * for pthread target
danielhrisca Dec 12, 2024
7514136
some performance tweaks
danielhrisca Dec 12, 2024
fef55c9
fast_path ?
danielhrisca Dec 12, 2024
2ba4427
optimize for performance
danielhrisca Dec 12, 2024
99b3375
control the thread count for get_channel_raw_bytes_parallel and data_…
danielhrisca Dec 13, 2024
98b694b
remove the 2min limit
danielhrisca Dec 13, 2024
f6f971a
getattr and setattr
danielhrisca Dec 13, 2024
e6052eb
compile libdeflate
danielhrisca Dec 20, 2024
44d9ffe
64 bit fseek and ftell
danielhrisca Dec 20, 2024
9ec2342
fix free and arg parsing
danielhrisca Dec 20, 2024
2e43ccc
submodule
danielhrisca Dec 20, 2024
b8fea02
recursive
danielhrisca Dec 20, 2024
bcebc7c
submodules actions
danielhrisca Dec 20, 2024
ef80f21
ghilimele
danielhrisca Dec 20, 2024
7ff1b68
space
danielhrisca Dec 20, 2024
e9e7ef7
SUBMODULE_PAT
danielhrisca Dec 20, 2024
c617d4e
fix for *nix
danielhrisca Dec 20, 2024
9cb6159
if defined
danielhrisca Dec 20, 2024
5512e8d
fseeko
danielhrisca Dec 20, 2024
bd6f5eb
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
danielhrisca Dec 20, 2024
4690dae
10% better transpose
danielhrisca Dec 21, 2024
94333d0
before optim
danielhrisca Dec 26, 2024
5eaad49
optimized ?:?
danielhrisca Dec 26, 2024
3e74974
winapi mmap
danielhrisca Dec 26, 2024
9e92437
mmap
danielhrisca Dec 27, 2024
5d76e4d
style
danielhrisca Dec 27, 2024
70b80cd
single access
danielhrisca Jan 3, 2025
c76a0d1
barbaric adaptation for the select function
danielhrisca Jan 3, 2025
56630b0
use Fragment class instead of tuple
danielhrisca Jan 8, 2025
6adceeb
next
danielhrisca Jan 10, 2025
e192767
linux fix
danielhrisca Jan 13, 2025
ff79e38
Py_NewRef not available in py3.9
danielhrisca Jan 13, 2025
8b26ce3
linux imports
danielhrisca Jan 13, 2025
dde3299
double parenthesis
danielhrisca Jan 13, 2025
9f0185c
use the new Fragment objects
danielhrisca Jan 13, 2025
930305b
more fragment usage
danielhrisca Jan 13, 2025
39238d0
only works for mapped file
danielhrisca Jan 13, 2025
120b4f2
MDF v4 only
danielhrisca Jan 13, 2025
d5bd27d
add libdefalte license notice
danielhrisca Jan 14, 2025
0b0d7f3
prevent \n avalanche
danielhrisca Jan 14, 2025
9103225
disable get_channel_raw_bytes_complete usage for now
danielhrisca Jan 14, 2025
b8873d2
fixes #1127 : add process_bus_logging to stack and concatenate
danielhrisca Jan 14, 2025
c49dacd
default values for Fragment arguments
danielhrisca Jan 14, 2025
6b284f8
handle LIN scale ranges
danielhrisca Jan 14, 2025
d254e6c
improve LDF channel unit in case of scale ranges
danielhrisca Jan 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ jobs:

steps:
- uses: actions/checkout@v4
with:
submodules: 'recursive'
token: ${{ secrets.SUBMODULE_PAT }}

- name: Set up Python
uses: actions/setup-python@v5
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/coveralls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ jobs:

steps:
- uses: actions/checkout@v4
with:
submodules: 'recursive'
token: ${{ secrets.SUBMODULE_PAT }}

- name: Set up Python
uses: actions/setup-python@v5
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ jobs:

steps:
- uses: actions/checkout@v4
with:
submodules: 'recursive'
token: ${{ secrets.SUBMODULE_PAT }}

- name: Set up Python
uses: actions/setup-python@v5
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ jobs:

steps:
- uses: actions/checkout@v4
with:
submodules: 'recursive'
token: ${{ secrets.SUBMODULE_PAT }}

# Used to host cibuildwheel
- uses: actions/setup-python@v5
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ dist/
downloads/
eggs/
.eggs/
lib/
/lib/
lib64/
parts/
sdist/
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "ext/libdeflate"]
path = ext/libdeflate
url = https://github.com/ebiggers/libdeflate
26 changes: 14 additions & 12 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,24 +1,26 @@
cmake_minimum_required(VERSION 3.15...3.26)

project(
${SKBUILD_PROJECT_NAME}
LANGUAGES C
VERSION ${SKBUILD_PROJECT_VERSION})
cmake_minimum_required(VERSION 3.26...3.29)
project(${SKBUILD_PROJECT_NAME} LANGUAGES C VERSION ${SKBUILD_PROJECT_VERSION})

find_package(
Python
REQUIRED
COMPONENTS
Interpreter
Development.Module
${SKBUILD_SABI_COMPONENT}
NumPy
REQUIRED)
)

set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(LIBDEFLATE_COMPRESSION_SUPPORT OFF)
set(LIBDEFLATE_GZIP_SUPPORT OFF)
set(LIBDEFLATE_BUILD_GZIP OFF)

# Add submodule libdeflate
add_subdirectory(ext/libdeflate EXCLUDE_FROM_ALL)

python_add_library(cutils
MODULE
src/asammdf/blocks/cutils.c
WITH_SOABI USE_SABI 3.9)
python_add_library(cutils MODULE WITH_SOABI USE_SABI 3.9 src/asammdf/blocks/cutils.c)

target_link_libraries(cutils PRIVATE Python::NumPy)
target_link_libraries(cutils PRIVATE Python::NumPy libdeflate::libdeflate_static)

install(TARGETS cutils DESTINATION "asammdf/blocks")
31 changes: 31 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -163,3 +163,34 @@ whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

--------------------------------------------------------------------------------

3rdparty dependency libdeflate is statically linked.
libdeflate has the following license:


Copyright 2016 Eric Biggers
Copyright 2024 Google LLC

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation files
(the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

--------------------------------------------------------------------------------
1 change: 1 addition & 0 deletions ext/libdeflate
Submodule libdeflate added at 780519
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ skip = ["pp*", "*_ppc64le", "*-musllinux*", "*_s390x"]

[tool.ruff]
target-version = "py39"
extend-exclude = ["./src/asammdf/gui/ui"]
extend-exclude = ["./src/asammdf/gui/ui", "./ext"]
force-exclude = true

[tool.ruff.lint]
Expand Down
3 changes: 2 additions & 1 deletion src/asammdf/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from .blocks.utils import load_channel_names_from_file
from .gui import plot
from .mdf import MDF, SUPPORTED_VERSIONS
from .signal import Signal
from .signal import InvalidationArray, Signal
from .version import __version__

try:
Expand All @@ -28,6 +28,7 @@
__all__ = [
"MDF",
"SUPPORTED_VERSIONS",
"InvalidationArray",
"Signal",
"Source",
"__cextension__",
Expand Down
28 changes: 25 additions & 3 deletions src/asammdf/blocks/bus_logging_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -396,10 +396,16 @@ def extract_mux(
sig_name = sig.name

try:
scale_ranges = getattr(sig, "scale_ranges", None)
if scale_ranges:
unit = scale_ranges[0]["unit"] or ""
else:
unit = sig.unit or ""

signals[sig_name] = {
"name": sig_name,
"comment": sig.comment or "",
"unit": sig.unit or "",
"unit": unit,
"samples": samples if raw else apply_conversion(samples, sig, ignore_value2text_conversion),
"conversion": get_conversion(sig) if raw else None,
"t": t_,
Expand Down Expand Up @@ -441,8 +447,24 @@ def get_conversion(signal: Signal) -> v4b.ChannelConversion | None:

a, b = float(signal.factor), float(signal.offset)

if signal.values:
conv = {}
conv = {}

scale_ranges = getattr(signal, "scale_ranges", None)
if scale_ranges:
for i, scale_info in enumerate(scale_ranges):
conv[f"upper_{i}"] = scale_info["max"]
conv[f"lower_{i}"] = scale_info["min"]
conv[f"text_{i}"] = from_dict({"a": scale_info["factor"], "b": scale_info["offset"]})

for i, (val, text) in enumerate(signal.values.items(), len(scale_ranges)):
conv[f"upper_{i}"] = val
conv[f"lower_{i}"] = val
conv[f"text_{i}"] = text

conv["default"] = from_dict({"a": a, "b": b})

elif signal.values:

for i, (val, text) in enumerate(signal.values.items()):
conv[f"upper_{i}"] = val
conv[f"lower_{i}"] = val
Expand Down
Loading
Loading