Paper: AI driven WatermarkingTechnique for Safegaurding Text Integrity #928
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Atharva-Rasane
force-pushed
the
scipy-2024-atharva_rasane
branch
from
c693614
to
c9e172b
Compare
Atharva-Rasane
force-pushed
the
scipy-2024-atharva_rasane
branch
from
c9e172b
to
43cd8f6
Compare
|
Hi @hongsupshin, I am not sure why the ID is returning null. Can you help please? |
|
Hey @Atharva-Rasane - thanks for the submission! The check is expecting your content at |
|
@Atharva-Rasane can you make the update requested here |
|
Curvenote Preview
|
|
Apologies for the delay, I have made the changes. Do you need me to squash the changes into 1 commit or is this fine now? |
|
this should be okay. thanks |
|
Hello @Atharva-Rasane I'm Tolulade, Data & ML Architect, founder @ Reispar Technologies. I'm happy to be one of the reviewers for your paper. |
|
@Atharva-Rasane @Tolulade-A please review and update the paper title with correct spelling and grammar. 🙂 |
Tolulade-A
suggested changes
Jun 12, 2024
There was a problem hiding this comment.
hi @Atharva-Rasane I have added some comments for you to review.
papers/atharva_rasane/main.md
Outdated
|
|
||
| In this paper, we explore text watermarking as a potential solution. We examine various methods, focusing on plain ASCII text in English. Our goal is to investigate different techniques, including physical watermarking (e.g., UniSpaCh by Por et al.), where text is modified to hide a binary message using Unicode Spaces, and logical watermarking (e.g., word context proposed by Jalil et al.), where a watermark key is generated via a defined process. While logical watermarking is difficult to break, it is not detectable without prior knowledge of the algorithm and parameters used. Conversely, physical watermarks are easily detected but also easy to break. | ||
|
|
||
| This paper presents a unique physical watermarking technique based on word substitution to address these challenges. The core idea is that AI models consistently produce the same output for the same input. Initially, we replaced every ith word with a "[MASK]," then used a BERT model to predict the most probable token in place of "[MASK]." The resulting text constitutes the watermarked text. To verify, we reran the algorithm on the watermarked text and compared the input and output for similarity. |
There was a problem hiding this comment.
@Atharva-Rasane you can add more context and clarity where the ith word is used, is this a random word sampling?
There was a problem hiding this comment.
Hello, yes it is a random index value. We could choose i-th to be say the 5-th word. If I added a hyphen, would that help a bit more with the clarity?
There was a problem hiding this comment.
Now I get you, adding a hypen would help and a bit of additional context like you did with this reply.
papers/atharva_rasane/main.md
Outdated
|
|
||
| This paper presents a unique physical watermarking technique based on word substitution to address these challenges. The core idea is that AI models consistently produce the same output for the same input. Initially, we replaced every ith word with a "[MASK]," then used a BERT model to predict the most probable token in place of "[MASK]." The resulting text constitutes the watermarked text. To verify, we reran the algorithm on the watermarked text and compared the input and output for similarity. | ||
|
|
||
| The Python implementation of the algorithm in this paper employes models from the HuggingFace Transformer Library, namely "bert-base-uncased" and "distilroberta-base". The "[MASK]" placeholder was generated by splitting the input string using the `split()` function and then replacing every ith element in the list with "[MASK]". This modified list served as the input text for the BERT model, where the output corresponding to each "[MASK]" was replaced accordingly. Finally, applying the join() function to the list produces the watermarked text. |
There was a problem hiding this comment.
@Atharva-Rasane the ith word is also used here, kindly add more context, is this a random word sampling ?
papers/atharva_rasane/main.md
Outdated
|
|
||
| With the majority of the internet and tools like ChatGPT and Bard being text-focused, we need to realize the importance of identifying the source of text whether due to copyright or to differentiate between AI-generated text and Human written text to prevent the flow of misinformation. The standard of detecting AI-generated text is with the use of another ML classifier which needs to be constantly trained on the latest AI-generated text data. This approach has a few drawbacks, one of which is the ever-changing nature of AI-generated text where we have bigger and better models that are giving more human-like text being released faster then ever before and thus we need a more standard/concrete approach, one that can be used regardless of the AI model i.e. we need a method of identifying that doesn't depend on one generating the text. One such approach is via the use of a watermark. | ||
|
|
||
| Watermarks are an identifying pattern used to identify the origin of the data. In this case, we specifically want to focus on text watermarking (watermarking of plain text). Text watermarking can broadly be classified into 2 types Logical Embedding and Physical Embedding which in turn can be classified further. Logical Embedding involves the user generating a watermark key by some logic from the input text. Note that this means that the input text is not altered and the user instead keeps the generated watermark key to identify the text. Physical Embedding involves the user altering the input text itself to insert a message into it and the user instead runs an algorithm to find this message to identify the text. In this paper, we will propose an algorithm to watermark text using BERT (Bidirectional Encoder Representations from Transformers), a model introduced by Google whose main purpose is to replace a special symbol [MASK] with the most probable word given the context. |
There was a problem hiding this comment.
@Atharva-Rasane is MASK here an acronymn? If it is, it can be spelt in full.
There was a problem hiding this comment.
MASK here is not an acronym but signifies a special token which BERT model then identifies and replaces - This token and its usage was pulled directly from one of the cited papers.
There was a problem hiding this comment.
did I miss it ? Was there any place you elaborated slightly on what Mask is ? This could help simplify things.
There was a problem hiding this comment.
I have incuded it in the inroduction:
"In this paper, we will propose an algorithm to watermark text using BERT (Bidirectional Encoder Representations from Transformers), a model introduced by Google whose main purpose is to replace a special symbol "[MASK]" with the most probable word given the context."
There was a problem hiding this comment.
Hi @Atharva-Rasane I have taken note of the changes, but I will still go through them again and get back to you if there's any other feedback
Atharva-Rasane
force-pushed
the
scipy-2024-atharva_rasane
branch
from
886ef5c
to
33e4b62
Compare
Atharva-Rasane
force-pushed
the
scipy-2024-atharva_rasane
branch
from
33e4b62
to
37b266e
Compare
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
| # Ensure that this title is the same as the one in `myst.yml` | ||
| title: AI-Driven Watermarking Technique for Safeguarding Text Integrity in the Digital Age | ||
| abstract: | | ||
| The internet's growth has led to a surge in text usage. Now, with public access to generative AI models like ChatGPT/Bard, identifying the source is vital. This is crucial due to concerns about copyright infringement and plagiarism. Moreover, it's essential to differentiate AI-generated text to curb misinformation from AI model hallucinations. |
There was a problem hiding this comment.
Please replace "it's" with "it is". "It is" is formal.
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
| --- | ||
|
|
||
| ## Introduction | ||
| The growth of the internet is driven by the spread of web pages, which are written in HTML (Hyper Text Markup Language). These web pages contain large amounts of text. Almost every webpage in some form or another contains text making it a popular mode of communication whether it be blogs, posts, articles, comments etc. Text can be represented as a collection of ASCII or Unicode values, where each value corresponds to a specific character. Given the text-focused nature of the internet and tools like ChatGPT and Bard, it is crucial to identify the source of text. This helps to manage copyright issues and distinguish between AI-generated and human-written text, thereby preventing the spread of misinformation. Currently, detecting AI-generated text relies on machine learning classifiers that need frequent retraining with the latest AI-generated data. However, this method has drawbacks, such as the rapid evolution of AI models producing increasingly human-like text. Therefore, a more stable approach is needed, one that does not depend on the specific AI model generating the text. |
There was a problem hiding this comment.
ChatGPT or Bard, (and not "and") unless other tools are specifically ignored here. In that case, remove "like".
There was a problem hiding this comment.
I have Replaced and with or
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
| ## Introduction | ||
| The growth of the internet is driven by the spread of web pages, which are written in HTML (Hyper Text Markup Language). These web pages contain large amounts of text. Almost every webpage in some form or another contains text making it a popular mode of communication whether it be blogs, posts, articles, comments etc. Text can be represented as a collection of ASCII or Unicode values, where each value corresponds to a specific character. Given the text-focused nature of the internet and tools like ChatGPT and Bard, it is crucial to identify the source of text. This helps to manage copyright issues and distinguish between AI-generated and human-written text, thereby preventing the spread of misinformation. Currently, detecting AI-generated text relies on machine learning classifiers that need frequent retraining with the latest AI-generated data. However, this method has drawbacks, such as the rapid evolution of AI models producing increasingly human-like text. Therefore, a more stable approach is needed, one that does not depend on the specific AI model generating the text. | ||
|
|
||
| Watermarks are an identifying pattern used to trace the origin of the data. In this case, we specifically want to focus on text watermarking (watermarking of plain text). Text watermarking can broadly be classified into 2 types Logical Embedding and Physical Embedding which in turn can be classified further [@Atr01]. Logical Embedding involves the user generating a watermark key by some logic from the input text. Note that this means that the input text is not altered and the user instead keeps the generated watermark key to identify the text. Physical Embedding involves the user altering the input text itself to insert a message into it and the user instead runs an algorithm to find this message to identify the text. In this paper, we will propose an algorithm to watermark text using BERT (Bidirectional Encoder Representations from Transformers), a model introduced by Google whose main purpose is to replace a special symbol "[MASK]" with the most probable word given the context. |
There was a problem hiding this comment.
... into 2 types, Logical Embedding and Physical Embedding, ...
Please add commas for readability.
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
| ## Proposed Model | ||
| "BERT-based watermarking is based on the 5-gram approach by Lancaster[@Atr02]. However, our focus is on watermarking any text, regardless of its origin. This paper will use **bert-base-uncased** model, which finds the most probable uncased English word to replace the [MASK] token. | ||
|
|
||
| Note that a different variant of BERT can be trained on different language datasets and thus will generate a different result and as such the unique identity to consider here is the BERT model i.e. if the user wants a unique watermark they need to train/develop the BERT model on their own. This paper is not concerned with the type of BERT model and is focused on its conceptual application for watermarking. Thus for us, BERT is a black box model that returns the most probable word given the context with the only condition being that it has a constant temperature i.e. it doesn't hallucinate (produce different results for the same input). For our purposes, you can think of the proposed algorithm as a many to one function which is responsible for converting the input text into a subset of watermarked set. |
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
| templ = tempd[0] | ||
| temps = templ['token_str'] | ||
| watermarked_words[i+4] = temps.split()[0] | ||
| # print("Done ", i + 1, "th word") |
There was a problem hiding this comment.
I have removed the commented code
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
| # Example usage | ||
| text = "Quantum computing is a rapidly evolving field that leverages the principles of quantum mechanics to perform computations that are infeasible for classical computers. Unlike classical computers, which use bits as the fundamental unit of information, quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously due to the principles of superposition and entanglement, providing a significant advantage in solving complex computational problems." | ||
| # match_ratios = watermark_text_and_calculate_matches(text, max_offset=5) | ||
| # check_significant_difference(match_ratios) |
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
|
|
||
| # Calculate match ratios | ||
| match_ratios = watermark_text_and_calculate_matches(text, max_offset=5) | ||
| result = {0: 0.5384615384615384, 1: 0.6153846153846154, 2: 0.5833333333333334, 3: 0.6666666666666666, 4: 0.5833333333333334} |
There was a problem hiding this comment.
Consider rounding the decimals to fewer digits.
sanhitamj
reviewed
Jun 30, 2024
papers/atharva_rasane/main.md
Outdated
| (result) modified_text = "Quantum computing is example a rapidly evolving field that leverages the principles of quantum mechanics to perform random computations that are impossible for classical computers. Unlike quantum computers, which use bits as the random insert fundamental unit of , quantum computers use quantum bits or qubits. Qubits can exist in multiple states simultaneously according random to the principles of symmetry and entanglement, providing a significant advantage in solving complex mathematical problems." | ||
|
|
||
| match_ratios = watermark_text_and_calculate_matches(modified_text, max_offset=5) | ||
| (result) match_ratios = {0: 0.5714285714285714, 1: 0.5714285714285714, 2: 0.5384615384615384, 3: 0.38461538461538464, 4: 0.7692307692307693} |
There was a problem hiding this comment.
Consider rounding the decimals to fewer digits.
papers/atharva_rasane/main.md
Outdated
| Highest Match Ratio: 0.7692307692307693 | ||
| Average of Other Ratios: 0.5164835164835164 | ||
| T-Statistic: -5.66220858504931 | ||
| P-Value: 0.010908789440745323 |
There was a problem hiding this comment.
Consider rounding the decimals to fewer digits.
| Highest Match Ratio: 0.7692307692307693 | ||
| Average of Other Ratios: 0.5164835164835164 | ||
| T-Statistic: -5.66220858504931 | ||
| P-Value: 0.010908789440745323 |
There was a problem hiding this comment.
As above - Consider rounding the decimals to fewer digits.
There was a problem hiding this comment.
I have round all down to 2 decimal places
papers/atharva_rasane/main.md
Outdated
| 4. Potential loss of context: When words are replaced, the intended context of delivery could be altered. However, AI models are continually improving, and we hope that a well-trained model can significantly mitigate this risk. | ||
|
|
||
| **Real-world applicability:** | ||
| 1. Versatility in applications: This method can be applied across various fields such as copyright protection, and content authentication, and in legal and academic settings where proof of authorship is crucial. It's particularly beneficial for managing copyrights in digital media, academic papers, and any online content where text is dynamically generated or reused. |
There was a problem hiding this comment.
I replaced all contractions in the paper
…esn't and more with their full forms
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If you are creating this PR in order to submit a draft of your paper, please name your PR with
Paper: AI driven WatermarkingTechnique for Safegaurding Text Integrity. An editor will then add apaperlabel and GitHub Actions will be run to check and build your paper.See the project readme for more information.
Editor: Sanhita Joshi @sanhitamj
Reviewers: