Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verify accuracy of current srapers #10

Open
sethetter opened this issue Jan 19, 2017 · 10 comments
Open

Verify accuracy of current srapers #10

sethetter opened this issue Jan 19, 2017 · 10 comments

Comments

@sethetter
Copy link
Member

Non-technical friendly!

Verify the dates that are generated by the scraper for the meetings they are scraping.

The different meeting types are listed here, and the corresponding dates for the meetings are here. The dates correspond to the types by matching tid to the type's id.

If you see any issues, say so here! If it looks good, also say so here!

@Mearnest
Copy link
Collaborator

I will attempt to verify the information and dates on what I have so far, but any help would be awseome! I am but one pair of eyes.

@Mearnest Mearnest self-assigned this Jan 23, 2017
@Mearnest
Copy link
Collaborator

Actually, best leave this to someone else.

@Mearnest Mearnest removed their assignment Jan 23, 2017
@kaylavix
Copy link

kaylavix commented Apr 20, 2017

So, is there a way for me to know what was scraped and what was just input here by a human https://github.com/openwichita/public-meetings/blob/master/priv/data/meeting_types.json? Or do we need both input types to be checked?

@aaronarduino
Copy link
Collaborator

@kaylavix I think all the data comes from the scrapers. So I think all needs to be checked. @Mearnest can correct me if I'm wrong about that.

@Mearnest
Copy link
Collaborator

Everything but the Goat and Pandas meetings, which were just testing manual entry on the website.

@kaylavix
Copy link

kaylavix commented Apr 20, 2017

id 1 looks good. That City Council pdf also included 2018 meetings for January, but those aren't in there.
id 2 is also good. Another 2018 date that wasn't scraped?

@kaylavix
Copy link

kaylavix commented Apr 20, 2017

Something's up with id 3, the Board of County Commissioners meetings. Dates are ahead by a couple days. Checked it using the url in the data: https://sedgwickcounty.legistar.com/Calendar.aspx

I'm also not sure how we got dates past April 19, because I can't find anything past that in the above page. Not even if I use their "export to Excel doc" option. Unless it's just assuming Wednesdays.

@kaylavix
Copy link

For id 4, Board of Education for 259, I don't see a meeting for March 27, but it's in the data ("id": "113").

There also needs to be a meeting on 6/29. It's their year-end reports meeting.

@Mearnest
Copy link
Collaborator

@kaylavix 2018 data wasn't scraped on purpose. It was just meant to be yearly scraping. I added a test 2018 meeting just because.

Also, it's odd to be responding to this 2 months laters. Whoops.

@sethetter
Copy link
Member Author

sethetter commented Jun 19, 2017 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants