Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server entity database #350

Merged
merged 27 commits into from
Oct 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
687bfed
Moving all data into the server database
Wavesonics Sep 5, 2024
c59cabb
Client is now sending the project server ID
Wavesonics Sep 11, 2024
e30166c
More work on client and server to support UUIDs
Wavesonics Sep 12, 2024
4891cdf
Implemented data migrator
Wavesonics Sep 14, 2024
d03dbe8
Disable Kover
Wavesonics Sep 14, 2024
2fa63d3
Wow things seem to be working?
Wavesonics Sep 17, 2024
dcbe359
Use FALSE instead of 0
Wavesonics Sep 20, 2024
7a1c4e8
Format
Wavesonics Sep 20, 2024
d5e1719
Fixed Database migration
Wavesonics Sep 20, 2024
eec623e
Consolidate serialization version
Wavesonics Sep 20, 2024
30b8bae
Enable Kover
Wavesonics Sep 20, 2024
7af376f
Disable Kover again
Wavesonics Sep 20, 2024
b98347b
Implemented server entity deletions
Wavesonics Sep 22, 2024
d651e07
Projects Sync now properly sets the project UUID when creating
Wavesonics Sep 23, 2024
bda8c5a
Fixed project delete failing locally
Wavesonics Sep 23, 2024
1b25026
Project and Entity deletion is now working!
Wavesonics Sep 24, 2024
650251f
Changed entity type from into to string in DB
Wavesonics Sep 27, 2024
7eb0aa7
Fixing up server tests
Wavesonics Sep 27, 2024
4945b00
Implemented first Projects sync E2E test
Wavesonics Sep 29, 2024
a906804
Project Sync - Golden Path test
Wavesonics Sep 29, 2024
903b81d
Project Sync - Golden Path test!
Wavesonics Sep 30, 2024
8e845af
refactored Project sync test to make more tests easier
Wavesonics Sep 30, 2024
a5d2aa8
E2E sync test now does uploading and downloading!
Wavesonics Oct 1, 2024
2ed098f
Fix build
Wavesonics Oct 2, 2024
198c937
More Server sync tests
Wavesonics Oct 2, 2024
0096e2c
JVM 21
Wavesonics Oct 2, 2024
9404f40
JVM 21
Wavesonics Oct 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,21 +13,23 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: set up JDK 17
- name: set up JDK 21
uses: actions/setup-java@v4
with:
java-version: '17'
java-version: '21'
distribution: 'temurin'
cache: gradle

- name: Grant execute permission for gradlew
run: chmod +x gradlew
- name: Build with Gradle
run: ./gradlew buildDebug
- name: Run Kover
run: ./gradlew koverXmlReport
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4
# - name: Run Kover
# run: ./gradlew koverXmlReport
# - name: Upload coverage reports to Codecov
# uses: codecov/codecov-action@v4
- name: Run Server Migration Tests
run: ./gradlew :server:verifySqlDelightMigration
# ios:
# runs-on: macos-latest
# steps:
Expand All @@ -40,10 +42,10 @@ jobs:
# - name: Get swift version
# run: swift --version
#
# - name: set up JDK 17
# - name: set up JDK 21
# uses: actions/setup-java@v4
# with:
# java-version: '17'
# java-version: '21'
# distribution: 'temurin'
# cache: gradle
#
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4

- name: set up JDK 17
- name: set up JDK 21
uses: actions/setup-java@v4
with:
java-version: '17'
java-version: '21'
distribution: 'temurin'
cache: gradle

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publishInternal.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ jobs:
steps:
- uses: actions/checkout@v4

- name: set up JDK 17
- name: set up JDK 21
uses: actions/setup-java@v4
with:
java-version: '17'
java-version: '21'
distribution: 'temurin'
cache: gradle

Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
uses: actions/setup-java@v4
with:
distribution: 'adopt'
java-version: 17
java-version: 21

- name: Gradle caching
uses: actions/cache@v4
Expand Down Expand Up @@ -78,7 +78,7 @@ jobs:
uses: actions/setup-java@v4
with:
distribution: 'adopt'
java-version: 17
java-version: 21

- name: Gradle caching
uses: actions/cache@v4
Expand Down Expand Up @@ -131,7 +131,7 @@ jobs:
uses: actions/setup-java@v4
with:
distribution: 'adopt'
java-version: 17
java-version: 21

- name: Gradle caching
uses: actions/cache@v4
Expand Down Expand Up @@ -184,7 +184,7 @@ jobs:
uses: actions/setup-java@v4
with:
distribution: 'adopt'
java-version: 17
java-version: 21

- name: Gradle caching
uses: actions/cache@v4
Expand Down Expand Up @@ -239,7 +239,7 @@ jobs:
uses: actions/setup-java@v4
with:
distribution: 'adopt'
java-version: 17
java-version: 21

- name: Gradle caching
uses: actions/cache@v4
Expand Down
28 changes: 28 additions & 0 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,34 @@ scope.launch(defaultDispatcher) {
}
```

## Logging

## Client

On the client you can log using `Napier` it works on all supported platforms:

```kotlin
Napier.i("message")
Napier.w("message")
Napier.e("message")
Napier.d("message")
```

## Server

On server you can log anywhere you have access to the ktor `Application`

```kotlin
log.info("message")
log.debug("message")
```

You can also access it from a ktor `Call` object:
`call.application.environment.log.info("Hello from a Call!")`

If you need logging below the HTTP layer 🤷 Pass the logger down? Idk we don't have a great solution
for this yet.

## Synchronization

The protocol for synchronizing data between client and server is outlined here:
Expand Down
10 changes: 3 additions & 7 deletions base/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
import org.jetbrains.kotlin.gradle.dsl.JvmTarget

plugins {
alias(libs.plugins.kotlin.multiplatform)
alias(libs.plugins.kotlin.serialization)
Expand All @@ -17,15 +15,13 @@ repositories {

kotlin {
androidTarget()
jvm("desktop") {
compilerOptions {
jvmTarget.set(JvmTarget.fromTarget(libs.versions.jvm.get()))
}
}
jvm("desktop")
iosX64()
iosArm64()
iosSimulatorArm64()

jvmToolchain(libs.versions.jvm.get().toInt())

applyDefaultHierarchyTemplate()

sourceSets {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
package com.darkrockstudios.apps.hammer.base

import korlibs.io.util.UUID
import kotlinx.serialization.Serializable
import kotlin.jvm.JvmInline

@Serializable
@JvmInline
value class ProjectId(val id: String) {
companion object {
fun randomUUID() = ProjectId(UUID.randomUUID().toString())
}
}

Original file line number Diff line number Diff line change
Expand Up @@ -67,16 +67,24 @@ sealed interface ApiProjectEntity {
val content: String
) : ApiProjectEntity

enum class Type {
SCENE,
NOTE,
TIMELINE_EVENT,
ENCYCLOPEDIA_ENTRY,
SCENE_DRAFT;
enum class Type(val id: Int) {
SCENE(0),
NOTE(1),
TIMELINE_EVENT(2),
ENCYCLOPEDIA_ENTRY(3),
SCENE_DRAFT(4);

fun toStringId() = when (this) {
SCENE -> "scene"
NOTE -> "note"
TIMELINE_EVENT -> "timeline_event"
ENCYCLOPEDIA_ENTRY -> "encyclopedia_entry"
SCENE_DRAFT -> "scene_draft"
}

companion object {
fun fromString(string: String): Type? {
return when (string.trim().lowercase()) {
fun fromString(string: String?): Type? {
return when (string?.trim()?.lowercase()) {
"scene" -> SCENE
"note" -> NOTE
"timeline_event" -> TIMELINE_EVENT
Expand All @@ -85,6 +93,10 @@ sealed interface ApiProjectEntity {
else -> null
}
}

fun fromInt(id: Int?): Type? {
return entries.find { it.id == id }
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
package com.darkrockstudios.apps.hammer.base.http

const val AUTH_REALM = "User Resources"

const val HAMMER_PROTOCOL_HEADER = "X-Hammer-Protocol-Version"
const val HAMMER_PROTOCOL_VERSION = 0
const val HAMMER_PROTOCOL_VERSION = 1
const val HEADER_SERVER_VERSION = "X-Server-Version"
const val HEADER_CLIENT_VERSION = "X-Client-Version"
const val HEADER_SYNC_ID = "X-Sync-Id"
const val HEADER_ENTITY_HASH = "X-Entity-Hash"
const val HEADER_ORIGINAL_HASH = "X-Original-Hash"
const val HEADER_ENTITY_TYPE = "X-Entity-Type"
const val HEADER_ENTITY_TYPE = "X-Entity-Type"

const val API_ROUTE_PREFIX = "api"
Original file line number Diff line number Diff line change
@@ -1,10 +1,23 @@
package com.darkrockstudios.apps.hammer.base.http

import com.darkrockstudios.apps.hammer.base.ProjectId
import kotlinx.serialization.Serializable

@Serializable
data class ApiProjectDefinition(
val name: String,
val uuid: ProjectId,
)

@Serializable
data class BeginProjectsSyncResponse(
val syncId: String,
val projects: Set<String>,
val deletedProjects: Set<String>,
)
val projects: Set<ApiProjectDefinition>,
val deletedProjects: Set<ProjectId>,
)

@Serializable
data class CreateProjectResponse(
val projectId: ProjectId,
val alreadyExisted: Boolean,
)
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,9 @@ class AccountSettingsComponent(

override fun removeServer() {
globalSettingsRepository.deleteServerSettings()
projectsRepository.getProjects().forEach { projectDef ->
projectsRepository.removeProjectId(projectDef = projectDef)
}
}

override suspend fun setAutomaticBackups(value: Boolean) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import com.darkrockstudios.apps.hammer.common.components.projectselection.Projec
import com.darkrockstudios.apps.hammer.common.components.savableState
import com.darkrockstudios.apps.hammer.common.components.storyeditor.metadata.ProjectMetadata
import com.darkrockstudios.apps.hammer.common.data.ProjectDef
import com.darkrockstudios.apps.hammer.common.data.SyncedProjectDefinition
import com.darkrockstudios.apps.hammer.common.data.globalsettings.GlobalSettingsRepository
import com.darkrockstudios.apps.hammer.common.data.isSuccess
import com.darkrockstudios.apps.hammer.common.data.projectmetadatarepository.ProjectMetadataDatasource
Expand Down Expand Up @@ -221,10 +222,17 @@ class ProjectsListComponent(
}

override fun deleteProject(projectDef: ProjectDef) {
val projectId = projectsRepository.getProjectId(projectDef)
val syncedProject = if (projectId != null) {
SyncedProjectDefinition(projectDef, projectId)
} else {
null
}

if (projectsRepository.deleteProject(projectDef)) {
Napier.i("Project deleted: ${projectDef.name}")
if (projectsSynchronizer.isServerSynchronized()) {
projectsSynchronizer.deleteProject(projectDef)
if (syncedProject != null) {
projectsSynchronizer.deleteProject(syncedProject)
}

loadProjectList()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package com.darkrockstudios.apps.hammer.common.components.storyeditor.metadata

import com.darkrockstudios.apps.hammer.base.ProjectId
import kotlinx.datetime.Instant
import kotlinx.serialization.Serializable

Expand All @@ -16,5 +17,6 @@ data class ProjectMetadata(
data class Info(
val created: Instant,
val lastAccessed: Instant? = null,
val dataVersion: Int = 0 // Default to 0, if we don't know the version, assume its the oldest
)
val dataVersion: Int = 0, // Default to 0, if we don't know the version, assume its the oldest
val serverProjectId: ProjectId? = null,
)
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
package com.darkrockstudios.apps.hammer.common.data

import com.darkrockstudios.apps.hammer.base.ProjectId
import com.darkrockstudios.apps.hammer.common.fileio.HPath
import kotlinx.serialization.Serializable

@Serializable
data class ProjectDefinition(val name: String, val path: HPath)

typealias ProjectDef = ProjectDefinition
typealias ProjectDef = ProjectDefinition

@Serializable
data class SyncedProjectDefinition(val projectDef: ProjectDefinition, val projectId: ProjectId)
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ abstract class IdRepository(private val projectDef: ProjectDef) : ProjectScoped
private var nextId: Int = -1

fun peekNextId(): Int = nextId
fun peekLastId(): Int = nextId - 1

suspend fun findNextId() {
mutex.withLock {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package com.darkrockstudios.apps.hammer.common.data.projectmetadatarepository

import com.darkrockstudios.apps.hammer.base.ProjectId
import com.darkrockstudios.apps.hammer.common.components.storyeditor.metadata.ProjectMetadata
import com.darkrockstudios.apps.hammer.common.data.ProjectDef
import com.darkrockstudios.apps.hammer.common.fileio.HPath
Expand All @@ -14,4 +15,8 @@ abstract class ProjectMetadataDatasource(
abstract fun saveMetadata(metadata: ProjectMetadata, projectDef: ProjectDef)
abstract fun updateMetadata(projectDef: ProjectDef, block: (metadata: ProjectMetadata) -> ProjectMetadata)
abstract fun getMetadataPath(projectDef: ProjectDef): HPath
}

fun ProjectMetadataDatasource.loadProjectId(projectDef: ProjectDef): ProjectId {
return loadMetadata(projectDef).info.serverProjectId ?: error("Server project ID missing")
}
Loading