Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IO issues for 'io.network_from_csv' #2959

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open

Fix IO issues for 'io.network_from_csv' #2959

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9ec5384
Fix IO issues for 'io.network_from_csv'
hpc-dave Nov 11, 2024
df66574
Fix linter error in _csv.py
hpc-dave Feb 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions openpnm/io/_csv.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import re

import numpy as np
from pandas import read_table
from pandas import read_csv

from openpnm.io import _parse_filename
from openpnm.io._pandas import network_to_pandas, project_to_pandas
Expand Down Expand Up @@ -44,12 +44,11 @@ def network_from_csv(filename):
from openpnm.network import Network
fname = _parse_filename(filename=filename, ext='csv')

a = read_table(filepath_or_buffer=fname,
sep=',',
skipinitialspace=True,
index_col=False,
true_values=['T', 't', 'True', 'true', 'TRUE'],
false_values=['F', 'f', 'False', 'false', 'FALSE'])
a = read_csv(filepath_or_buffer=fname,
sep=',',
skipinitialspace=True,
header=0,
low_memory=False)

# First parse through all the items and re-merge columns
dct = {}
Expand Down Expand Up @@ -87,6 +86,15 @@ def network_from_csv(filename):
if k.startswith('throat.'):
dct[k] = v[:Nt, ...]

# This is a fix for pandas loading in all data types as dtype('O')
# if it encounters mixed types in the file. For whatever reason
# the conversion to type 'float64' works without problem, however for
# boolean values, the dtype 'O' persists. That can lead to difficult
# to debug errors later on. Therefore here this fix:
for k, v in dct.items():
if type(v[0]) is bool:
dct[k] = v.astype(bool)

network = Network()
network.update(dct)
return network
Loading