Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A working API to automate the creation of questions #765

Merged
merged 35 commits into from
Sep 12, 2024

Conversation

ombhardwajj
Copy link
Contributor

What

A working API to automate the creation of questions for the all the tables of "subject types" and "Address" has been successfully built.

How does it work?

  1. User sends a POST req at /api/metabase/create-questions
  2. It calls "createQuestionsForSubjectTypes" method
  3. This method first waits for the Database Sync to complete in metabase and then begins with the main task.
  4. First it searches and stores all the names of the "subject types" in a List (since the tables would have the same name).
  5. Then it executes the main POST req with payload to metabase which would create the questions/cards

@ombhardwajj ombhardwajj changed the title avniproject/avni-server#762 | A working API to automate the creation of questions avniproject#762 | A working API to automate the creation of questions Jul 24, 2024
…te the creation of questions for the tables subject types and Address
@ombhardwajj ombhardwajj changed the title avniproject#762 | A working API to automate the creation of questions A working API to automate the creation of questions Jul 24, 2024
@ombhardwajj ombhardwajj force-pushed the autoquess branch 2 times, most recently from b7fccd3 to 77bee27 Compare August 11, 2024 18:18
Copy link
Member

@vinayvenu vinayvenu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DatabaseRepository - Create proper objects for what you return, not JsonNode.

DatabaseService - Don't return Strings if there are specific known values. Move them into enum values

DatabaseService (createQuestionForTable) - Push towards making smaller objects that describe your data instead of directly using ObjectNodes.

DatabaseService - Eventually, we will plug this in when there are changes in programs, subject types etc. Which means there needs to be methods to make changes for specific subject types

StringUtils.java - There is an S.java that is meant for String utilities

getDatabaseDetails is being called multiple times. This is a network call, and can potentially get slow. Try to optimize

Not sure of the purpose of the db migrations

application.properties changes need to be reverted

Need to check how this works.
int tableId = getTableIdByName("Subject Type");
When running first time, the subject type tables will be of the form "schema"."subjectTypeName", so how do the subject type tables get created?


public enum TableType {
INDIVIDUAL("Individual"),
HOUSEHOLD("Household"),
Copy link
Contributor

@himeshr himeshr Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is table type for "Encounter".?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have "ProgramEncounter" isn't it? 🤔

@PostMapping("/create-questions")
public void createQuestions() throws Exception{
databaseService.createQuestionsForSubjectTypes();
databaseService.createQuestionsForProgramsAndEncounters();
Copy link
Contributor

@himeshr himeshr Sep 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Define and invoke databaseService.createQuestionsForSubjectGeneralEncounterTypes()

@ombhardwajj ombhardwajj force-pushed the autoquess branch 3 times, most recently from d6dbf4a to e9ff297 Compare September 7, 2024 08:05
…al tables : Its now using AddressQuestionCreationService
@himeshr
Copy link
Contributor

himeshr commented Sep 9, 2024

@ombhardwajj Pretty happy with the overall design and functionality of the Automated Question Creation service on Metabase.

Have specified list of code review comments from our code walkthrough session earlier:

  • Rename createQuestionForIndividualTable to non Individual something, DatabaseRepository
  • For Enums and other constants created to represent Metabase entityTypes, provide documentation links.
  • Metbase client jar is available, for Java8 avni version and If easy, only then replace..
  • Delete MetabaseJoin.java
  • Use ConditionType.equal in MetabaseQueryBuilder => conditionArray
  • Replace "base-type" with Enums in MetabaseQueryBuilder
  • Create constants wherever demmed fit MetabaseQueryBuilder
  • Link doc to MetabaseRequestBody
  • See if TableDetails is needed in fetchingTableDetails calls from Metabase
  • TableType.Encounter and its questions creation
  • Move AddressQuestionCreationService logic to DatabaseService
  • Replace getSubjectTypeNames usage of int literals with constants depictiing columnsNames
  • Create common method called by createQuestionsForSubjectTypes() and createQuestionsForPrograms(), createQuestionsForProgramsEncounters() and createQuestionsForEncounters() with List entityNames
  • createQuestionForTable( schema => !public is not used)

…ed createQuestionForIndividualTable method , Deleted MetabaseJoin
@himeshr
Copy link
Contributor

himeshr commented Sep 10, 2024

@ombhardwajj another thought that occurred to me regarding this, in-case the Questions were previously created and we introduce a new SubjectType / EncounterType, etc.. if we retrigger the AutoQuestionCreationAPI "/create-questions", would it work as expected and create Question only for the new one and not try and recreate all of them.?
Ideally, Question creation should be an Idempotent operation.

…tabaseService 2) Replaced getSubjectTypeNames usage of int literals with constants depictiing columnsNames 3)Created common method called by createQuestionsForSubjectTypes() and createQuestionsForPrograms(), createQuestionsForProgramsEncounters() and createQuestionsForEncounters() with List entityNames 4)Added CollectionItem , Now before creating questions, the existing ones are filtered out
@ombhardwajj
Copy link
Contributor Author

Hey @himeshr I've addressed all the concerns including duplication of questions. Please review the code 🙇

@himeshr
Copy link
Contributor

himeshr commented Sep 12, 2024

"createQuestionForTable( schema => !public is not used)" => This one is pending, but going ahead with merge as its a small inconsequential thing.

@himeshr himeshr merged commit 837a48c into avniproject:master Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants