tips for running on google colab

[sujee]

Why are these changes needed?

To enable run DPK applications on google colab.

Related issue number (if any).

#582

[sujee]

need to know where we can link to this document

[daw3rd]

Where should this file be linked from?
Also fdedupe to fedeup, for consistency

[sujee]

Also fdedupe to fedeup, for consistency

You mean this? These are program args for the fuzzy transform.

    # infrastructure
    "fdedup_bucket_cpu": 0.3,
    "fdedup_doc_cpu": 0.3,
    "fdedup_mhash_cpu": 0.3,
    "fdedup_num_doc_actors": 1,
    "fdedup_num_bucket_actors": 1,
    "fdedup_num_minhash_actors": 1,
    "fdedup_num_preprocessors": 1,

[sujee]

Where should this file be linked from?

May be we can create a Tips and Troubleshooting section in the main README

I can use some input on this : @Bytes-Explorer @shahrokhDaijavad

[shahrokhDaijavad]

@sujee There are other Tips and Troubleshooting issues dispersed in various doc files in the repo (e.g., mac.md and memory.md files in the same place you have put your google-colab.md file). If we create such a section in the README file, it's better to consolidate all of them into one md file with different sections for different tips. I think at some point soon, we should do this, but for now, I think we should just add a link to a section of README file that I submitted a PR for earlier this morning (PR #593) (to be reviewed by Hima). In the new README, I have put your Google Colab example before setting the local environment and we can add a sentence there saying something like: Though you won't need them for this simple example, here are some tips for running on Google Coalb and add the link

[sujee]

@shahrokhDaijavad I like this idea 👍

[Bytes-Explorer]

yes a good idea, @sujee do you want to update the readme with tips for trouble shooting based on your experience

…

________________________________ From: Sujee Maniyam ***@***.***> Sent: 17 September 2024 01:20 To: IBM/data-prep-kit ***@***.***> Cc: Hima Patel ***@***.***>; Mention ***@***.***> Subject: [EXTERNAL] Re: [IBM/data-prep-kit] tips for running on google colab (PR #587) @ shahrokhDaijavad I like this idea 👍 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned. Message ID: <IBM/data-prep-kit/pull/587/c2353785050@ github. com> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ @shahrokhDaijavad<https://github.com/shahrokhDaijavad> I like this idea 👍 — Reply to this email directly, view it on GitHub<#587 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANKCJ6X6TNJCVUAA6EIX7FTZW4Y77AVCNFSM6AAAAABOE4QOTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJTG44DKMBVGA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[daw3rd]

Also fdedupe to fedeup, for consistency

You mean this? These are program args for the fuzzy transform.

    # infrastructure
    "fdedup_bucket_cpu": 0.3,
    "fdedup_doc_cpu": 0.3,
    "fdedup_mhash_cpu": 0.3,
    "fdedup_num_doc_actors": 1,
    "fdedup_num_bucket_actors": 1,
    "fdedup_num_minhash_actors": 1,
    "fdedup_num_preprocessors": 1,

No, i saw doc references to fededupe which is something new. I think you mean fdedup. Let's not introduce new terms

[shahrokhDaijavad]

LGTM