Skip to content

Scrape fixing #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 82 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
f84e789
[MAJOR-3]
AstrophysicsAndPython Aug 2, 2022
4b4444b
[MAJOR-1]
DamnUi Aug 7, 2022
4db75f4
[really minor1]
DamnUi Aug 7, 2022
5ffe74e
[really-minor2]
DamnUi Aug 7, 2022
427f4e5
kinda-MAJOR2
DamnUi Aug 7, 2022
167818b
[really-Minor-3]
DamnUi Aug 7, 2022
ccca7b0
[Really-Minor-4]
DamnUi Aug 7, 2022
9ea3094
[Really-Minor5]
DamnUi Aug 7, 2022
0698ba0
[MINOR3]
DamnUi Aug 7, 2022
93b9a05
[MINOR-4]
AstrophysicsAndPython Aug 7, 2022
3c7d37a
[MAJOR2]
DamnUi Aug 7, 2022
e3607bb
[MAJOR-3]
DamnUi Aug 7, 2022
410bdfe
[MINOR6?]
DamnUi Aug 7, 2022
35e693b
[MINOR-7]
DamnUi Aug 8, 2022
72dd7bf
[MINOR-8]
DamnUi Aug 8, 2022
abfaf0e
[Really-MINOR-6]
DamnUi Aug 8, 2022
711a411
[Really-MINOR-7]
DamnUi Aug 8, 2022
93342fa
[MINOR-9]
DamnUi Aug 8, 2022
4ab372d
Update PyStats.py
DamnUi Aug 8, 2022
877b80f
[NEEDS TO BE HEAVILY REVISED I DONT KNOW WHAT SHIT I HAVE DONE]
DamnUi Aug 8, 2022
7bb0138
[MINOR-10]
AstrophysicsAndPython Aug 8, 2022
a078427
Update README.md
DamnUi Aug 9, 2022
cf06f74
Revert "[MINOR-10]"
DamnUi Aug 9, 2022
d62b410
[MINOR-11]
AstrophysicsAndPython Aug 10, 2022
7b646ef
Merge branch 'main' into scrap-dev2
AstrophysicsAndPython Aug 10, 2022
c7efd05
[MINOR-12]
DamnUi Aug 14, 2022
5404e72
[Really-Minor-IDC]
DamnUi Aug 15, 2022
74fa358
[MINOR-13]
AstrophysicsAndPython Aug 15, 2022
d5b056f
[MINOR-14]
DamnUi Aug 15, 2022
b884c1e
[MAJOR-4]
DamnUi Aug 16, 2022
4f28eb2
[MAJOR-5]
DamnUi Aug 16, 2022
c1730cf
Create func and class.py
DamnUi Aug 16, 2022
d2de55b
[MINOR-15]
DamnUi Aug 16, 2022
3a8b6b9
[MAJOR-6]
AstrophysicsAndPython Aug 16, 2022
9caba9a
[testing_commit]
AstrophysicsAndPython Aug 16, 2022
f2da9df
[MINOR-16]
AstrophysicsAndPython Aug 17, 2022
8935a35
[MINOR-18]
DamnUi Aug 17, 2022
f7a2944
[MINOR-17]
AstrophysicsAndPython Aug 17, 2022
1663f58
[MINOR-18]
AstrophysicsAndPython Aug 17, 2022
6491906
[MINOR-19]
AstrophysicsAndPython Aug 17, 2022
eceb213
[MAJOR-7]
DamnUi Aug 17, 2022
00c3493
[MINOR-20]
DamnUi Aug 17, 2022
608d95a
[MINOR-21]
DamnUi Aug 18, 2022
37d9272
WHAT IS THIS SHIT I WROTE IDK but it works
DamnUi Aug 19, 2022
f8bed48
[2nd commit here]
DamnUi Aug 19, 2022
31b30ee
Update PyStats.py
DamnUi Aug 19, 2022
4c56b3c
[MINOR-no]
DamnUi Aug 20, 2022
3c74f22
Update PyStats.py
DamnUi Aug 20, 2022
598148b
Update PyStats.py
DamnUi Aug 20, 2022
a1f8a5d
[MINOR-no2]
DamnUi Aug 21, 2022
a151c16
Update PyStats.py
DamnUi Aug 21, 2022
fd5ebcf
Merge pull request #7 from DamnUi/most_shit_code_ever_for_sure
DamnUi Aug 21, 2022
12e4f34
[Major-7]
DamnUi Aug 22, 2022
b85e38b
[MINOR-21]
DamnUi Aug 23, 2022
b12dbea
Update PyStats.py
DamnUi Aug 23, 2022
897967c
[MINOR-22]
DamnUi Aug 24, 2022
285ce02
Update PyStats.py
DamnUi Aug 24, 2022
da27460
[MINOR-23]
DamnUi Aug 24, 2022
7adbf02
[]
DamnUi Aug 24, 2022
f52085c
Autofix issues in 5 files
deepsource-autofix[bot] Aug 24, 2022
fa43cd9
Delete checks.py
DamnUi Aug 24, 2022
625d941
Merge branch 'scrap-dev2' of https://github.com/DamnUi/PyStats into s…
DamnUi Aug 24, 2022
1db06b8
[MINOR-24]
DamnUi Aug 24, 2022
b864ae1
Fixed typos
norbcodes Aug 25, 2022
4a4d581
Merge pull request #9 from norbcodes/scrap_dev2
DamnUi Aug 25, 2022
d40a25c
Revert "Fixed typos"
DamnUi Aug 25, 2022
67fa8c9
Merge pull request #10 from DamnUi/revert-9-scrap_dev2
DamnUi Aug 25, 2022
839922e
[MINOR-25]
DamnUi Aug 26, 2022
afa2ba9
Merge branch 'scrap-dev2' of https://github.com/DamnUi/PyStats into s…
DamnUi Aug 26, 2022
03f0f9b
[MINOR-24]
AstrophysicsAndPython Aug 26, 2022
b584ce1
Merge branch 'scrap-dev2' of https://github.com/DamnUi/PyStats into s…
DamnUi Aug 27, 2022
9d22614
[maybe-MAJOR-8]
DamnUi Aug 27, 2022
8dbfe89
[MINOR-25]
DamnUi Aug 27, 2022
e8dbe9f
[MINR-26]
DamnUi Aug 27, 2022
0c6b0ef
[MINOR-27]
DamnUi Aug 27, 2022
55f4246
[MINOR-28]
AstrophysicsAndPython Aug 27, 2022
a2caecd
[MAJOR-9]
AstrophysicsAndPython Aug 27, 2022
c4bc01b
[MINOR-29]
AstrophysicsAndPython Aug 27, 2022
b3be98e
[MINOR-28]
DamnUi Aug 27, 2022
7b03ca5
[MINOR-30]
AstrophysicsAndPython Aug 27, 2022
4c27676
[MAJOR-10]
AstrophysicsAndPython Aug 27, 2022
2bffc8f
Update README.md
DamnUi Aug 31, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -131,3 +131,4 @@ dmypy.json

# testing files
/run_pystats.bash
/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@
from rich.markdown import Markdown
from rich.align import Align
from rich.layout import Layout
from datetime import datetime

console = Console(height=5) #possibly some kind of external algorithm to find an appropriate size and length?

maintitle_markdown = f"""
# PyStats
# PyStats {datetime.time(datetime.now())}
"""


Expand All @@ -23,7 +24,7 @@



stat_less_md = f"""
stat_less_md = """
File Path(s):? , Pc_Name: not needed , User: not needed , Date: , Time_to_file_report: possibly not needed
could also add info about varibles and imports here?
"""
Expand All @@ -39,20 +40,37 @@
wraped_less_stats = Align(Panel(stat_less_md, title='Quickies', title_align='left', width=91), align='left', style='black')#Can Change border style here by changing style

#Now for the real main screen
layout = Layout()
def make_bar(size_1=50, size_2=85):
layout = Layout()

layout.split_column(
Layout(name="lower", ratio=5)
)
layout.split_column(
Layout(name="lower", ratio=5)
)


layout["lower"].split_row(
Layout(name="left"),
Layout(name="right"),
)

layout["lower"]["left"].size = size_1
layout['lower']['right'].size = size_2

return layout['lower']

bar1 = make_bar()
bar2 = make_bar()
bar3 = make_bar()

layout["lower"].split_row(
Layout(name="left"),
Layout(name="right"),
unwrap = Group(
Markdown('Text', style='red', justify='center')


)

layout["lower"]["left"].size = 50



console.print(wraped_main_screen, wraped_less_stats, layout)
bar1['left'].update(Panel(unwrap, style='blue'))

console.print(wraped_less_stats, bar1, bar2, bar3)
161 changes: 18 additions & 143 deletions PyStats.py
Original file line number Diff line number Diff line change
@@ -1,143 +1,18 @@
import argparse
# Dashboard imports
import os
from collections import Counter

from rich import pretty, print
# Fine path called d:\1.Autohotkey\.Python\Project Folders\portswwww.py
from rich.traceback import install as install_traceback

# Usage: print(Panel.fit("Hello, [red]World!", title="Welcome", subtitle="Thank you"))
pretty.install()
install_traceback(show_locals=False)
# Dashboard imports \\ or pretty imports?

# Initializing Parser

if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("-df", help="Input Absolute Path to Directory or File")
# df is directory or file it will kinda figure out itself assuming theirs only 2 files in
# the dir and u do -neglect
parser.add_argument('-onefile', help="Input Absolute Path to File to Analyze")
parser.add_argument("-neglect", help="Input Absolute Path to File to Ignore ONLY IN DIRECTORY")
# Debug argument to print out the file names
path = parser.parse_args()
# 2 Variables
# path.df
# path.onefile ignores everything and makes sure u have just onefile ur scanning
# Path.neglect

if path.onefile:
paths = [path.onefile]
else:
# Determine if it's a directory or file
if os.path.isdir(path.df):
# If it's a directory get all absolute paths of files in it ending with py
paths = []
for dir_path, _, filenames in os.walk(path.df):
for f in filenames:
if f.endswith('.py'):
paths.append(os.path.abspath(os.path.join(dir_path, f)))
# remove neglect from paths
if path.neglect is not None:
for line in paths:
original_line = line
line = line.replace("\\", "/")
if path.neglect in line:
paths.remove(original_line)
else:
paths = [path.df]
# That ends classification of the file


class Stat:
def __init__(self, directory) -> None:
self.directory = directory
# Add .py to directory if not there
if not self.directory[-1].endswith('.py'):
self.directory[-1] += '.py'
# Check if file exists
if not os.path.exists(self.directory[-1]):
print("[red]File does not exist")
exit()
# The Neglect keyword ain't needed because it's already delt in the if statement from before

def scrape_imports(self):
imports_ = []
real_imports = []
if len(self.directory) > 1:
print("[green]Scraping imports from multiple files...")
for path in self.directory:
with open(path, encoding='utf8') as f:
for line in f:
if line.startswith('import'):
imports_.append(line)
elif line.startswith('from'):
imports_.append(line)
else:
print("[green]Scraping imports from single file...")
with open(self.directory[0], encoding='utf8') as f:
for line in f:
if line.startswith('import'):
imports_.append(line)
elif line.startswith('from'):
imports_.append(line)

for line in imports_:
real_imports.append(line.strip())

return real_imports

def line_count(self):
# if it's a directory return the lines in a dict with the file name as the key
if len(self.directory) > 1:
line_count = {}
for path in self.directory:
with open(path, encoding='utf8') as f:
line_count[path] = len(f.readlines())
else:
with open(self.directory[0], encoding='utf8') as f:
line_count = len(f.readlines())

return line_count

def most_used_variable(self):
variables = self.scrape_variables()
variable_count = Counter(variables)
dictionary = {}
for key, value in variable_count.items():
dictionary[key] = value
most_used_variable = max(dictionary, key=dictionary.get)
return most_used_variable, dictionary[most_used_variable]

def scrape_variables(self, full_line_of_variable=False):
variables = []
var = {}
for i in self.directory:
with open(i, encoding='utf8') as f:
for line in f:
if ' = ' in line:
if full_line_of_variable:
variables.append(line)
variables.append(line.split()[0])

return variables

def get_import_names(self):
imports = self.scrape_imports()
import_names = []
for line in imports:
if line.startswith('import'):
import_names.append(line.split()[1])
elif line.startswith('from'):
import_names.append(line.split()[3])

return import_names


info = Stat(paths)

print(info.line_count(),
info.most_used_variable(),
info.scrape_variables())
from rich.console import Console

import _PyStats

console = Console(record=True)
# Changed it to pystat_print because it wouldn't let me print other things that I wanted using
# the standard print statement
pystat_print = console.print

if __name__ == "__main__":
info = _PyStats.VisualWrapper(_PyStats.working_path)
if _PyStats.args.adhd:
info = _PyStats.VisualWrapper(_PyStats.working_path)

if _PyStats.args.imgpath:
info.img_render(remove_check=False, force_show=True, clear_screen=True)
else:
pystat_print(info.get_all())
16 changes: 11 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,17 @@
# PyStats
![PStats](https://user-images.githubusercontent.com/81849260/185852278-b154d1db-b7a4-412d-bc5d-cd17b1ae754f.png)
## Intro
PyStats scrapes your Python files for any data it can find and presents it intuitively.

Project in semi-beta
This Branch is currently the most updated afaik

Things to do currently:
Add a dashboard to view all info in - semidone
## Quick Start
Currently not on pypi so none of this is going to work YET
```py
pip install PyStat?
```


![carbon](https://user-images.githubusercontent.com/81849260/185844371-d31146a5-27eb-40d6-a433-d7ae034bb3f5.png)


Thank you so much https://github.com/AstrophysicsAndPython
Special Thanks to [Astro](https://github.com/AstrophysicsAndPython)
Loading