-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Raw Data #1
Comments
Would be great to see a sample structure, as the output from my discovergy smartmeters needs to be converted! |
i´m also interested ina a sample file! I would have a File with more than 70k Input Lines to test the script. |
According to https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html the default delimiter assumed is ','. |
As this issue was targeted at my Fork, the example for the fork is uploaded here: |
@foscoj @Habedere @lmoscattini I've included sample data in the repo and updated the notebooks to explain the data structure. Hope this helps, even though my reply is quite late. |
Is it possible to use a custom report from Fronius Solarweb Premium? As far as I see, only energy (kWh) is exported but only power for the different phases? See this file (header row is in German): |
@kiryph Looking at the data it seems like you already have a battery installed (e.g. column "Energie aus Batterie bezogen"). Are you trying to calculate how much more energy you could use from your PV system with a larger battery? As you mentioned, the file also contains the power for the different phases (colums "Wirkleistung L1/2/3 Einspeisepunkt | PowerMeter"). The sum of these thee columns should be the total power that you were providing to the grid or taking from the grid at a given time. I noticed that your data has a time resolution of 5 minutes so sharp power peaks might be averaged out. But I think that the script should still give you meaningful results. |
@PV-Soft Thanks for the quick response.
Yes, I have already installed a battery with a capacity of 5.12kWh. I am wondering whether it is financially sensible to upgrade the battery pack by one module (2.56 kWh). The Fronius Solarweb Battery Simulator also does not provide an option for extending the system by one module of 2.56kWh. One can only choose the smallest battery pack 5.12kWh (here the same as I actually have): So I cannot use this tool to simulate the smallest possible battery upgrade. Only 2x additional modules which the pictures shows with an additional self-consumption of 130kWh. The next size (3x additional battery modules) already means only an even more minor improvement by 18kWh self-consumption during more than 6 months compared to two additional modules (148kWh improvement compared to the existing battery pack).
I was assuming this. I will try it. I checked out also the fork https://github.com/foscoj/Battery-Simulation which uses a slightly different csv file, in particular a similarly formatted date and time string (German locale). My modified Jupyter notebook cell for reading the csv file: dateTimeFormatString = '%d.%m.%Y %H:%M' #German Time Format String, needed for Discovergy .csv
data = pd.read_csv(r'Solarkraftwerk_01112023-21062024.csv', encoding='latin-1', header=[0,1])
df2 = pd.DataFrame.from_records(data, columns = ['Datum und Uhrzeit', 'Wirkleistung L1 Einspeisepunkt | PowerMeter','Wirkleistung L2 Einspeisepunkt | PowerMeter', 'Wirkleistung L3 Einspeisepunkt | PowerMeter'])
df2['timestamp'] = df2['Datum und Uhrzeit']
df2['power'] = df2['Wirkleistung L1 Einspeisepunkt | PowerMeter'] + df2['Wirkleistung L2 Einspeisepunkt | PowerMeter'] + df2['Wirkleistung L3 Einspeisepunkt | PowerMeter']
df = pd.DataFrame.from_records(df2, columns = ['timestamp', 'power'])
df.tail() The next cell "Calculate Power and Energy usable by Battery" runs without issues. However, the cell "Run Battery Simulation" throws an error. Since I am not proficient with Pandas, maybe you know what the error tells me: Simulating battery of size 1 kWh.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
[/var/folders/2z/bljkcy1j2cbf7l9zf3dk0nqc0000gn/T/ipykernel_61892/219161310.py](http://localhost:57644/var/folders/2z/bljkcy1j2cbf7l9zf3dk0nqc0000gn/T/ipykernel_61892/219161310.py) in ?()
1 # Go through all batteries and start simulation for each one. Add results as columns to df.
2 for i in range(len(battery_sizes)):
----> 3 energy_in_battery, energy_excess_to_grid, power_provided_by_battery, total_power_provided_by_battery = simulate_battery(df, battery_sizes[i])
4 #battery_charge, battery_charge_excess, battery_discharge, battery_total_discharge = simulate_battery(df, battery_sizes[i])
5
6 df['energy_in_battery_' + battery_names[i]] = energy_in_battery #battery_charge
[/var/folders/2z/bljkcy1j2cbf7l9zf3dk0nqc0000gn/T/ipykernel_61892/1721990933.py](http://localhost:57644/var/folders/2z/bljkcy1j2cbf7l9zf3dk0nqc0000gn/T/ipykernel_61892/1721990933.py) in ?(df, battery_size)
16 # In this case, set battery state to battery_size or zero.
17 # Copy excess energy, not taken by battery to this_energy_excess_to_grid
18 this_energy_in_battery = row[1]['energy_battery'] + last_energy_in_battery
19 # If battery is full, copy excess to this_energy_excess_to_grid
---> 20 if this_energy_in_battery > battery_size:
21 this_energy_excess_to_grid = this_energy_in_battery - battery_size
22 this_energy_in_battery = battery_size
23 # If battery is empty, copy excess to this_energy_excess_to_grid
[~/Library/jupyterlab-desktop/jlab_server/lib/python3.12/site-packages/pandas/core/generic.py](http://localhost:57644/~/Library/jupyterlab-desktop/jlab_server/lib/python3.12/site-packages/pandas/core/generic.py) in ?(self)
1575 @final
1576 def __nonzero__(self) -> NoReturn:
-> 1577 raise ValueError(
1578 f"The truth value of a {type(self).__name__} is ambiguous. "
1579 "Use a.empty, a.bool(), a.item(), a.any() or a.all()."
1580 )
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all(). Maybe, a newer Pandas version is the issue.
Yes, the resolution is not as fine as yours. But I agree that the results should still be meaningful. |
I've loaded your sample file using following code: all_data = pd.read_csv('Detailed_Daily_Report_2024_05_11.csv', skiprows= [1])
all_data["timestamp"] = all_data["Datum und Uhrzeit"]
all_data["power"] = all_data["Wirkleistung L1 Einspeisepunkt | PowerMeter"] + all_data["Wirkleistung L2 Einspeisepunkt | PowerMeter"] + all_data["Wirkleistung L3 Einspeisepunkt | PowerMeter"]
all_data = all_data[["timestamp", "power"]]
all_data.head() In the following cell ("Calculate Power and Energy usable by Battery") I had to remove the argument Can you reproduce this? |
@PV-Soft the argument for the csv reader Now I have the data I was looking for.
The Fronius Simulator has returned 130kWh for the 5.12kWh size. But I think I have to reduce the sizes by 5% which should be the usable sizes and then matches very closely the result from Fronius (probably the same logic is used): Update Here the table with the battery sizes reduced by 5%:
For reference, I have also looked at https://github.com/hif2k1/battery_sim for Home Assistant but I am not sure if this can use historical data provided by a csv file at all. |
@kiryph I'm happy that the script helped you. The reason for the error was that your csv file contains two header rows. The second one is excluded with Unfortunately, I'm not familiar with the home assistant battery simulator that you mentioned, so I don't know if you can feed it with csv data. |
Hi Mate, do you happen to have a csv file for the raw data you used? If so, would you be able to post it, cheers!
The text was updated successfully, but these errors were encountered: