Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Searching for Available Commands by Voice #7

Open
Mark-Phillipson opened this issue Sep 14, 2020 · 2 comments
Open

Searching for Available Commands by Voice #7

Mark-Phillipson opened this issue Sep 14, 2020 · 2 comments

Comments

@Mark-Phillipson
Copy link

In addition to what can I say, it would be nice to have a command that you could for example say:

List Commands $SearchTerm$

And the system would return all commands with the search term contained within.

This because it's very difficult to remember the exact command name every time.

Also having to Press Alt+v every time defeats the use of speech recognition.

Ideally there should be a voice command to start and stop listening for example Listen and Stop Listening as an alternative.

@damian-666
Copy link

damian-666 commented Jul 19, 2023

Voice command and query , ui and code completion that is context aware in voice to intent... the indexed code /feauture seach in Dev Studio could be generalized fo any app/OS with complex UI in UI frameworks, or throuth ML and Auitomated Testintg via Automation. "AI first" LLMs+ Conttext awareness, domain specific vocabulary, an d command/Query Commpletion, should be generalized, apply to the OS as Start, ( lwould just add a mic , glows when you speak) chose any special mic by defaut that is not the mic array on a laptop ( makes all the difference to have a good speaking dynamic mic). Then each app starts the same. Any plugins imght handle special needs but this is a form of autocomplete that is hands free. could even scape llinux ands osx with the same.. peobably a third party wil do this, doenst see that 3 copilots is going the right waywith no hotwords. 4 year oldsurface sylus regression, bad UI, im now holding y breath. with this the bloated UI can be left as is an the AI system will bypass and or discover it as the best assistant.

@damian-666
Copy link

i fixed the typos in above, i
m used to the Ais reading right through them..people cant or wont.

typos are best fix in the fulll context.. not word by world. , in bulk.

regarding voice to text, even domain specific can help with that. but chap GP4 suggested to give it what the Voice access guessed i said,, then a screen capture of the menu i have open and it fixed the guess. to something on the menu.

for completion and running UI hands free or even no look, it can be amazing but i dont think this should be done specifically for dev studio, but all UI shouild be indexed, app object models exposed, and UI discoverable, it context used then laster command and query can be used without displaying the UI.
on the file menu i have only five or ten worlds i might say.. and hot words to switch to "Windows" or " Computer Start" otherwise obviously im not dictating . So this is a general problem and ideally works on mac , windows, and linux, and even uses OCR if it cannot figure out the via spy what the buttons tooltips and words are.. so voice assist is good but very teasing. its offline too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants