Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working with new version of TF & Extracted Hand class to separate file. #14

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

CT83
Copy link

@CT83 CT83 commented May 15, 2018

I plan to move more stuff around in the coming days, and refactor it so that different styling (biases, line height, width, colors) can be managed easily. I am planning to turn this into something beginner ML Engineers like me can easily work on.

Great work on the Repo!🎉

@CT83
Copy link
Author

CT83 commented May 15, 2018

I have also made few changes which allow the code to be compatible with current version of tensorflow. Please let me know what you think and if I should proceed with what I am doing and keep making PRs

@CT83 CT83 changed the title Extracted Hand class to separate file. Working with new version of TF & Extracted Hand class to separate file. May 16, 2018
@CT83
Copy link
Author

CT83 commented May 19, 2018

usage_demo
I am just expanding on your work here, but this is what I am trying to perfect. I will send my changes in a PR if you are interested.

@mandystroech
Copy link

Can you add more writing styles/fonts too?

@CT83
Copy link
Author

CT83 commented May 21, 2018

@mandystroech Head over to my Fork https://github.com/CT83/handwriting-synthesis .
See https://github.com/CT83/handwriting-synthesis/blob/testing/ct83_demo.py
Changing the styles/fonts is only a matter of changing the style, bias, line_height, stroke_widths you can get unlimited combinations using these. For anything else you would have to collect training data separately.

@TheMasterOfMagic
Copy link

@mandystroech Head over to my Fork https://github.com/CT83/handwriting-synthesis .
See https://github.com/CT83/handwriting-synthesis/blob/testing/ct83_demo.py
Changing the styles/fonts is only a matter of changing the style, bias, line_height, stroke_widths you can get unlimited combinations using these. For anything else you would have to collect training data separately.

Hi CT83,
I'm new to this, and since the original author seems to have disappeared for quite a long time, I was going to your fork to post this. But I cannot find the Issues entry there so I post this here. Could you please solve my questions below?

  1. Where do the style files come from? How to generate a new style?
  2. I have trained my own model following the instructions mentioned in the readme.md file, and my question is how do I use it? Is it relevant to question 1?

Best regards,
Newbie

@CT83
Copy link
Author

CT83 commented Nov 9, 2018

Hey @TheMasterOfMagic,

  1. I am probably in the same boat as you, I honestly don't know how the styles were generated, I will probably have to look into it myself, but I think maybe maybe you could try to view them using OpenCV's cv2.imshow() the styles seem to be numpy arrays and so are OpenCV images, I am not sure how that would work, but it's worth a try.
    You might have already read the paper this repo is based on Generating Sequences With Recurrent Neural Networks by Alex Graves does it shed any light on how this process works?

  2. You could try looking more into the rnn.py, tf_base_model.py and the checkpoints directory, that is where the original model seems to be stored.

I was working on this project more from refactoring the code point of view I wished I was of more help.
Also, I have just enabled issues on my Fork, you should see them now, but I would suggest keeping the issues here, as I more people will see those.

@TheMasterOfMagic
Copy link

Hey @CT83,

Thanks a lot !! And with poor English, here are my replies:

  • About the codes, I have traversed all those files and searched the keywords style and save, but it seems that these two words never appear at the same time in the project. That's what brings me here today.
  • I do know what .npy file is but I really didn't come up with cv2.imshow. Thanks for your in time reminding!
  • I also have read the paper roughly, and I'm going to read it more seriously, and I will keep a mind on this side. Once I get something I'll post it here :)

One more time, thanks a lot for your in time reply!

@TheMasterOfMagic
Copy link

Hi @CT83 , I got some findings about what those style files are!!
Last time, I np.load-ed those style files and still got nothing. It just seems that for each style, there is

  • a "characters" numpy file, which is a sentence in English
  • a "strokes" numpy file. which is a np.array with shape (n, 3). And for each raw, the first two values are always some small numbers around zero, and the third value is always a binary digit, mostly 0.

I used plt.imshow to show each strokes files but mess and dark is all I got. These days I've read the paper again and again, and I found something today! In the 3rd paragraph of subsection 4.1 Mixture Density Outputs on page 19, the paper says the first two values are "the pen offset from the previous input", and the third value defines if the current point ends a stroke(pen was lifted off the board).

I then calculated the cumulative sum of each strokes and I got images like below:

image

image

image

Each of them matches the sentence in their corresponding "characters" files. In other words, the sentences in the "character" files explain what the "strokes" files write.

But I still have no idea what the relationship between the NN and these style files is :(

@CT83
Copy link
Author

CT83 commented Nov 20, 2018

Wow! You made some progress! Awesome, keep at it.
I am glad to hear that the reading the paper helped.
You could try joining some online groups like Slack Channels those usually are full of smart people, who can help.
Google. Deep Learning Slack Groups

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants