-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Plugin: tap-planetscale #1684
Comments
I hit an issue with their tap (I was trying to write Meltano instructions to place in the planetscale readme) see https://github.com/planetscale/singer-tap/issues/51 |
The problem with having this in the Hub would be that |
@edgarrmondragon I have a memory of another non-python tap, but I don't think we added it to the Hub. Maybe @pnadolny13 remembers? It may have been this mysql one to be honest. |
There's https://github.com/MeltanoLabs/target-jsonl-blob which is also Golang-based, and we never added it to the Hub. |
@edgarrmondragon @tayloramurphy yeah I dont know of any that exist. I wonder if its helpful to still list it but exclude it from the API so its not accessible by meltano automatically. The hub page can explain how to install it manually if needed. I'd have to look into this but we could also list it as a docker airbyte variant https://hub.docker.com/r/planetscale/airbyte-source/tags. |
We could probably do something better than this but here's a "working" example. Note that their tap doesn't follow the singer spec so I don't think we can add this anyway :/ See https://github.com/planetscale/singer-tap/issues/51 maybe they'll adjust it though!
|
I played around a bit with the latest version of the planet scale tap (they fixed the catalog output) I can't get meltano to properly select the fields I have, or add the proper metadata. If I manually add a select, and update the replication-method to FULL_TABLE the tap works as advertised. Not sure exactly what's going on here, my best guess is the stream_id having a
Meltano.yml extractors:
- name: tap-planetscale
namespace: tap_planetscale
pip_url: cowsay
executable: /home/visch/git/tap-mysql/ps-singer-tap
capabilities:
- state
- catalog
- discover
settings:
- name: host
- name: username
- name: database
- name: password
sensitive: true
config:
host: aws.connect.psdb.cloud
username: 0fiqne6txvcqtjbdywan
database: tap-mysql
select:
- '*.*'
metadata:
'*':
replication-method: FULL_TABLE |
What is the plugin type?
extractor
Repository URL
https://github.com/planetscale/singer-tap/
Extension Repository URL
No response
Description
PlanetScale is the world’s most advanced MySQL platform
Domain URL
https://planetscale.com/docs
Logo Image
Any other notes?
This tap is written in Go.
Note that with some settings set automatically MeltanoLabs tap-mysql tap can support this as well (After MeltanoLabs/tap-mysql#45 is merged)
The text was updated successfully, but these errors were encountered: