Skip to content

Commit 8d51d11

Browse files
dmccallumdmccallum
committedMar 26, 2013
SSP-755 Initial commit
1 parent af21c6f commit 8d51d11

13 files changed

+1645
-1
lines changed
 

‎.gitignore

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
*.iml
2+
build
3+
.gradle
4+
.idea
5+
out

‎LICENSE

+202
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
2+
Apache License
3+
Version 2.0, January 2004
4+
http://www.apache.org/licenses/
5+
6+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7+
8+
1. Definitions.
9+
10+
"License" shall mean the terms and conditions for use, reproduction,
11+
and distribution as defined by Sections 1 through 9 of this document.
12+
13+
"Licensor" shall mean the copyright owner or entity authorized by
14+
the copyright owner that is granting the License.
15+
16+
"Legal Entity" shall mean the union of the acting entity and all
17+
other entities that control, are controlled by, or are under common
18+
control with that entity. For the purposes of this definition,
19+
"control" means (i) the power, direct or indirect, to cause the
20+
direction or management of such entity, whether by contract or
21+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
22+
outstanding shares, or (iii) beneficial ownership of such entity.
23+
24+
"You" (or "Your") shall mean an individual or Legal Entity
25+
exercising permissions granted by this License.
26+
27+
"Source" form shall mean the preferred form for making modifications,
28+
including but not limited to software source code, documentation
29+
source, and configuration files.
30+
31+
"Object" form shall mean any form resulting from mechanical
32+
transformation or translation of a Source form, including but
33+
not limited to compiled object code, generated documentation,
34+
and conversions to other media types.
35+
36+
"Work" shall mean the work of authorship, whether in Source or
37+
Object form, made available under the License, as indicated by a
38+
copyright notice that is included in or attached to the work
39+
(an example is provided in the Appendix below).
40+
41+
"Derivative Works" shall mean any work, whether in Source or Object
42+
form, that is based on (or derived from) the Work and for which the
43+
editorial revisions, annotations, elaborations, or other modifications
44+
represent, as a whole, an original work of authorship. For the purposes
45+
of this License, Derivative Works shall not include works that remain
46+
separable from, or merely link (or bind by name) to the interfaces of,
47+
the Work and Derivative Works thereof.
48+
49+
"Contribution" shall mean any work of authorship, including
50+
the original version of the Work and any modifications or additions
51+
to that Work or Derivative Works thereof, that is intentionally
52+
submitted to Licensor for inclusion in the Work by the copyright owner
53+
or by an individual or Legal Entity authorized to submit on behalf of
54+
the copyright owner. For the purposes of this definition, "submitted"
55+
means any form of electronic, verbal, or written communication sent
56+
to the Licensor or its representatives, including but not limited to
57+
communication on electronic mailing lists, source code control systems,
58+
and issue tracking systems that are managed by, or on behalf of, the
59+
Licensor for the purpose of discussing and improving the Work, but
60+
excluding communication that is conspicuously marked or otherwise
61+
designated in writing by the copyright owner as "Not a Contribution."
62+
63+
"Contributor" shall mean Licensor and any individual or Legal Entity
64+
on behalf of whom a Contribution has been received by Licensor and
65+
subsequently incorporated within the Work.
66+
67+
2. Grant of Copyright License. Subject to the terms and conditions of
68+
this License, each Contributor hereby grants to You a perpetual,
69+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70+
copyright license to reproduce, prepare Derivative Works of,
71+
publicly display, publicly perform, sublicense, and distribute the
72+
Work and such Derivative Works in Source or Object form.
73+
74+
3. Grant of Patent License. Subject to the terms and conditions of
75+
this License, each Contributor hereby grants to You a perpetual,
76+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77+
(except as stated in this section) patent license to make, have made,
78+
use, offer to sell, sell, import, and otherwise transfer the Work,
79+
where such license applies only to those patent claims licensable
80+
by such Contributor that are necessarily infringed by their
81+
Contribution(s) alone or by combination of their Contribution(s)
82+
with the Work to which such Contribution(s) was submitted. If You
83+
institute patent litigation against any entity (including a
84+
cross-claim or counterclaim in a lawsuit) alleging that the Work
85+
or a Contribution incorporated within the Work constitutes direct
86+
or contributory patent infringement, then any patent licenses
87+
granted to You under this License for that Work shall terminate
88+
as of the date such litigation is filed.
89+
90+
4. Redistribution. You may reproduce and distribute copies of the
91+
Work or Derivative Works thereof in any medium, with or without
92+
modifications, and in Source or Object form, provided that You
93+
meet the following conditions:
94+
95+
(a) You must give any other recipients of the Work or
96+
Derivative Works a copy of this License; and
97+
98+
(b) You must cause any modified files to carry prominent notices
99+
stating that You changed the files; and
100+
101+
(c) You must retain, in the Source form of any Derivative Works
102+
that You distribute, all copyright, patent, trademark, and
103+
attribution notices from the Source form of the Work,
104+
excluding those notices that do not pertain to any part of
105+
the Derivative Works; and
106+
107+
(d) If the Work includes a "NOTICE" text file as part of its
108+
distribution, then any Derivative Works that You distribute must
109+
include a readable copy of the attribution notices contained
110+
within such NOTICE file, excluding those notices that do not
111+
pertain to any part of the Derivative Works, in at least one
112+
of the following places: within a NOTICE text file distributed
113+
as part of the Derivative Works; within the Source form or
114+
documentation, if provided along with the Derivative Works; or,
115+
within a display generated by the Derivative Works, if and
116+
wherever such third-party notices normally appear. The contents
117+
of the NOTICE file are for informational purposes only and
118+
do not modify the License. You may add Your own attribution
119+
notices within Derivative Works that You distribute, alongside
120+
or as an addendum to the NOTICE text from the Work, provided
121+
that such additional attribution notices cannot be construed
122+
as modifying the License.
123+
124+
You may add Your own copyright statement to Your modifications and
125+
may provide additional or different license terms and conditions
126+
for use, reproduction, or distribution of Your modifications, or
127+
for any such Derivative Works as a whole, provided Your use,
128+
reproduction, and distribution of the Work otherwise complies with
129+
the conditions stated in this License.
130+
131+
5. Submission of Contributions. Unless You explicitly state otherwise,
132+
any Contribution intentionally submitted for inclusion in the Work
133+
by You to the Licensor shall be under the terms and conditions of
134+
this License, without any additional terms or conditions.
135+
Notwithstanding the above, nothing herein shall supersede or modify
136+
the terms of any separate license agreement you may have executed
137+
with Licensor regarding such Contributions.
138+
139+
6. Trademarks. This License does not grant permission to use the trade
140+
names, trademarks, service marks, or product names of the Licensor,
141+
except as required for reasonable and customary use in describing the
142+
origin of the Work and reproducing the content of the NOTICE file.
143+
144+
7. Disclaimer of Warranty. Unless required by applicable law or
145+
agreed to in writing, Licensor provides the Work (and each
146+
Contributor provides its Contributions) on an "AS IS" BASIS,
147+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148+
implied, including, without limitation, any warranties or conditions
149+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150+
PARTICULAR PURPOSE. You are solely responsible for determining the
151+
appropriateness of using or redistributing the Work and assume any
152+
risks associated with Your exercise of permissions under this License.
153+
154+
8. Limitation of Liability. In no event and under no legal theory,
155+
whether in tort (including negligence), contract, or otherwise,
156+
unless required by applicable law (such as deliberate and grossly
157+
negligent acts) or agreed to in writing, shall any Contributor be
158+
liable to You for damages, including any direct, indirect, special,
159+
incidental, or consequential damages of any character arising as a
160+
result of this License or out of the use or inability to use the
161+
Work (including but not limited to damages for loss of goodwill,
162+
work stoppage, computer failure or malfunction, or any and all
163+
other commercial damages or losses), even if such Contributor
164+
has been advised of the possibility of such damages.
165+
166+
9. Accepting Warranty or Additional Liability. While redistributing
167+
the Work or Derivative Works thereof, You may choose to offer,
168+
and charge a fee for, acceptance of support, warranty, indemnity,
169+
or other liability obligations and/or rights consistent with this
170+
License. However, in accepting such obligations, You may act only
171+
on Your own behalf and on Your sole responsibility, not on behalf
172+
of any other Contributor, and only if You agree to indemnify,
173+
defend, and hold each Contributor harmless for any liability
174+
incurred by, or claims asserted against, such Contributor by reason
175+
of your accepting any such warranty or additional liability.
176+
177+
END OF TERMS AND CONDITIONS
178+
179+
APPENDIX: How to apply the Apache License to your work.
180+
181+
To apply the Apache License to your work, attach the following
182+
boilerplate notice, with the fields enclosed by brackets "[]"
183+
replaced with your own identifying information. (Don't include
184+
the brackets!) The text should be enclosed in the appropriate
185+
comment syntax for the file format. We also recommend that a
186+
file or class name and description of purpose be included on the
187+
same "printed page" as the copyright notice for easier
188+
identification within third-party archives.
189+
190+
Copyright [yyyy] [name of copyright owner]
191+
192+
Licensed under the Apache License, Version 2.0 (the "License");
193+
you may not use this file except in compliance with the License.
194+
You may obtain a copy of the License at
195+
196+
http://www.apache.org/licenses/LICENSE-2.0
197+
198+
Unless required by applicable law or agreed to in writing, software
199+
distributed under the License is distributed on an "AS IS" BASIS,
200+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201+
See the License for the specific language governing permissions and
202+
limitations under the License.

‎NOTICE

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
Copyright 2013, JA-SIG, Inc.
2+
This project includes software developed by Jasig.
3+
http://www.jasig.org/
4+
5+
Licensed under the Apache License, Version 2.0 (the
6+
"License"); you may not use this file except in compliance
7+
with the License. You may obtain a copy of the License at:
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing,
12+
software distributed under the License is distributed on
13+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
KIND, either express or implied. See the License for the
15+
specific language governing permissions and limitations
16+
under the License.
17+
18+
This project includes:
19+
20+
Groovy under The Apache Software License, Version 2.0
21+
PostgreSQL JDBC Driver under BSD License
22+
jTDS under GNU Lesser General Public License (LGPL), Version 2.1

‎README.md

+109-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,112 @@
11
ssp-data-migration
22
==================
33

4-
Tool for moving SSP data from one data to another by representing that data as Liquibase changsets.
4+
Tool for moving SSP data from one data to another by representing that data as
5+
Liquibase changsets.
6+
7+
This tool will not touch your target database at all and interactions with
8+
the source database are strictly read-only.
9+
10+
There are no special provisions for locking the source database nor guaranteeing
11+
that it is in a read-only state. So you are probably best served to run this
12+
tool while the source application is not serving requests.
13+
14+
High-level workflow:
15+
16+
1. Configure a schema file describing the tables you want to export from the
17+
source database. The project includes a sample for SSP v1.2.0 at
18+
`src/main/groovy/resources/BaseConfigDBSchema` that would be suitable for
19+
migrating reference data types typically edited during a formal SSP
20+
training session.
21+
2. Run the tool, configured with that schema file and coordinates
22+
for your source database, redirecting stdout to an XML file.
23+
3. Take the resulting XML file and execute it against your target database using
24+
Liquibase tools. The simplest way to accomplish the latter is with the
25+
Liquibase [command line](http://www.liquibase.org/manual/command_line).
26+
27+
This is, of course, not the only way to move data from one database to another.
28+
PostgresSQL in particular ships with standard export/import tools which
29+
*should* be fully capable of migrating some or all SSP data between databases,
30+
thereby obviating this tool. But the situation has historically been more
31+
challenging with SqlServer, which has difficulty scripting the SSP database.
32+
Direct database-to-database export/import jobs have worked better, but these
33+
are often not allowed by network security policies.
34+
35+
Pre-requisites
36+
==============
37+
38+
At a minimum you'll need a Java JDK install. This process can vary widely from
39+
platform to platform, so you're on you're own for that one. This program will
40+
theoretically work on JDK 1.5+, but you'll be best served by 1.6+.
41+
42+
Depending on how you plan to run the application you can either install Groovy
43+
or Gradle. Gradle ships with an embedded Groovy, so you can technically avoid a
44+
standalone Groovy install if you'd like. If you just want to clone this repo and
45+
run the code with a minimum of fuss, Gradle is the way to go.
46+
47+
Gradle [download](http://www.gradle.org/downloads). The latest binary distro
48+
should be fine. We've most recently tested with 1.4.
49+
50+
Gradle [install instructions](http://www.gradle.org/docs/current/userguide/installation.html).
51+
The rest of this doc assumes you've added Gradle's `bin` directory to your PATH.
52+
53+
Groovy [download](http://groovy.codehaus.org/Download). The latest binary distro should be fine. The app has been
54+
tested with 1.8.6 and that's the version Gradle will bundle with the binary
55+
output.
56+
57+
Groovy [install instructions](http://groovy.codehaus.org/Installing+Groovy).
58+
The rest of this doc assumes you've added Groovy's `bin` directory to your PATH.
59+
60+
Running the Program
61+
===================
62+
63+
The app can be executed in at least three different ways:
64+
65+
1. Gradle run command. This is the easiest way to run the app if you are
66+
tweaking the source code:
67+
68+
`%> gradle -q run -PcliArgs="<opts>"`
69+
70+
Note that `<opts>` parsing is naive so spaces in argument values will not work
71+
as expected.
72+
73+
2. Exploded Gradle-assembled application
74+
75+
`%> gradle installApp`
76+
`%> ./build/install/ssp-data-migration/bin/ssp-data-migration <opts>`
77+
78+
`<opts>` splitting is handled by your shell in this case, so whitespace in
79+
argument values should work as expected.
80+
81+
Use this execution mechanism if you've received a zip or tar of the
82+
application.
83+
84+
3. Without Gradle. Don't know why you'd want to do this as it is significantly
85+
more verbose and you'll need to do more legwork to make your JDBC driver
86+
visible:
87+
88+
```
89+
%> groovy -cp "src/main/groovy/:src/main/resources:/path/to/your/jdbc/driver" \
90+
src/main/groovy/org/jasig/ssp/util/migration/Main.groovy \
91+
<opts>
92+
```
93+
94+
Output is an XML document on stdout. Redirect as appropriate.
95+
96+
Options all have POSIX "long opt" format, i.e. `--option-name=option-value`.
97+
The "=" is optional and can be replaced with whitespace. E.g.:
98+
99+
%> gradle -q run -PcliArgs="--db-url=jdbc:postgresql://localhost:5432/ssp" > migration.xml
100+
101+
Options
102+
=======
103+
104+
`--db-url` [Required] The full URL to the source database
105+
106+
`--db-username` [Optional] The username for the source database
107+
108+
`--db-password` [Optional] The password for the source database
109+
110+
`--schema-file` [Optional] Specify the location for the DB Schema file - the file that specifies what data to export (default: March 2013 tables from last export job)
111+
112+
`--usage`, `--help` Prints the usage message and performs no processing

‎build.gradle

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
/**
2+
* Licensed to Jasig under one or more contributor license
3+
* agreements. See the NOTICE file distributed with this work
4+
* for additional information regarding copyright ownership.
5+
* Jasig licenses this file to you under the Apache License,
6+
* Version 2.0 (the "License"); you may not use this file
7+
* except in compliance with the License. You may obtain a
8+
* copy of the License at:
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on
14+
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
apply plugin: 'eclipse'
20+
apply plugin: 'groovy'
21+
apply plugin:'application'
22+
23+
mainClassName = "org.jasig.ssp.util.migration.Main"
24+
25+
run {
26+
if(project.hasProperty("cliArgs")){
27+
args cliArgs.tokenize()
28+
}
29+
}
30+
31+
repositories {
32+
mavenCentral()
33+
}
34+
35+
dependencies {
36+
compile 'org.codehaus.groovy:groovy:1.8.6'
37+
compile 'postgresql:postgresql:9.1-901.jdbc4'
38+
compile 'net.sourceforge.jtds:jtds:1.2.4'
39+
}

‎src/main/groovy/org/jasig/ssp/util/migration/Main.groovy

+122
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
/**
2+
* Licensed to Jasig under one or more contributor license
3+
* agreements. See the NOTICE file distributed with this work
4+
* for additional information regarding copyright ownership.
5+
* Jasig licenses this file to you under the Apache License,
6+
* Version 2.0 (the "License"); you may not use this file
7+
* except in compliance with the License. You may obtain a
8+
* copy of the License at:
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on
14+
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
package org.jasig.ssp.util.migration
20+
21+
import static MigrationMeta.*
22+
23+
class Main {
24+
public static void main(String[] args) {
25+
26+
def processedArgs = handleOptions(args)
27+
28+
def dao = new MigrationTableObjectDAO(processedArgs)
29+
def xmlWriter = new MigrationXMLWriter(System.out, processedArgs)
30+
def xmlParser = new MigrationXMLParser(processedArgs)
31+
32+
xmlWriter.writeXMLOutput(dao.selectAllFromIteratedTables(xmlParser.parseDBSchemaFile()))
33+
}
34+
35+
/** Handles the options passed from the command line, prints warnings for
36+
* missing elements or exits the program if required options are missing.
37+
*
38+
* @param args the args passed from the command line
39+
* @return nothing
40+
*/
41+
private static def handleOptions(args) {
42+
43+
if (args.length > 0 && (
44+
args[0].contains(cliOption(HELP_FLAG)) ||
45+
args[0].contains(cliOption(USAGE_FLAG)))) {
46+
usage()
47+
System.exit(1)
48+
}
49+
50+
def processedArgs = [:]
51+
def expecting = null
52+
args.each { it ->
53+
if (expecting && !(isCliOption)) {
54+
processedArgs[expecting] = it
55+
return
56+
}
57+
58+
if (it.startsWith(cliOption(DB_URL_FLAG))) {
59+
if (hasValue(it, DB_URL_FLAG)) {
60+
processedArgs[DB_URL_FLAG] = valueOf(it, DB_URL_FLAG)
61+
} else {
62+
expecting = DB_URL_FLAG
63+
}
64+
} else if (it.startsWith(cliOption(SCHEMA_FILE_FLAG))) {
65+
if (hasValue(it)) {
66+
processedArgs[SCHEMA_FILE_FLAG] = valueOf(it, SCHEMA_FILE_FLAG)
67+
} else {
68+
expecting = SCHEMA_FILE_FLAG
69+
}
70+
} else if (it.startsWith(cliOption(DB_USERNAME_FLAG))) {
71+
if (hasValue(it)) {
72+
processedArgs[DB_USERNAME_FLAG] = valueOf(it, DB_USERNAME_FLAG)
73+
} else {
74+
expecting = DB_USERNAME_FLAG
75+
}
76+
} else if (it.startsWith(cliOption(DB_PASSWORD_FLAG))) {
77+
if (hasValue(it)) {
78+
processedArgs[DB_PASSWORD_FLAG] = valueOf(it, DB_PASSWORD_FLAG)
79+
} else {
80+
expecting = DB_PASSWORD_FLAG
81+
}
82+
}
83+
}
84+
85+
if (!(processedArgs[DB_URL_FLAG])) {
86+
sayErr "\nError: Database URL not set, please set this parameter using the ${cliOption(DB_URL_FLAG)} option"
87+
sayErr "(e.g.: ${cliOption(DB_URL_FLAG)}=jdbc:postgresql://localhost:5432/ssp)"
88+
usage()
89+
System.exit(1)
90+
}
91+
92+
if (!(processedArgs[SCHEMA_FILE_FLAG])) {
93+
sayErr "\nWarning: Database schema file (${cliOption(SCHEMA_FILE_FLAG)}) not set, defaulting to included db schema file (${DEFAULT_SCHEMA_FILE_NAME})."
94+
processedArgs[SCHEMA_FILE_FLAG] = defaultSchema();
95+
}
96+
97+
processedArgs
98+
}
99+
100+
private static def hasValue(cliArg, cliOpt) {
101+
cliArg ==~ "${cliOption(cliOpt)}=.*"
102+
}
103+
104+
private static def valueOf(cliArg, cliOpt) {
105+
(cliArg =~ "${cliOption(cliOpt)}=(.*)")[0][1]
106+
}
107+
108+
private static def defaultSchema() {
109+
MigrationXMLParser.class.getResourceAsStream(DEFAULT_SCHEMA_FILE_NAME)
110+
// println MigrationXMLParser.class.getResource(DEFAULT_SCHEMA_FILE_NAME)
111+
// new File(MigrationXMLParser.class.getResource(DEFAULT_SCHEMA_FILE_NAME).toURI()).getAbsolutePath();
112+
}
113+
114+
private static def sayErr(msg) {
115+
System.err.println(msg)
116+
}
117+
118+
private static def usage() {
119+
sayErr(USAGE)
120+
}
121+
122+
}

‎src/main/groovy/org/jasig/ssp/util/migration/MigrationMeta.groovy

+107
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
/**
2+
* Licensed to Jasig under one or more contributor license
3+
* agreements. See the NOTICE file distributed with this work
4+
* for additional information regarding copyright ownership.
5+
* Jasig licenses this file to you under the Apache License,
6+
* Version 2.0 (the "License"); you may not use this file
7+
* except in compliance with the License. You may obtain a
8+
* copy of the License at:
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on
14+
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
package org.jasig.ssp.util.migration
20+
21+
class MigrationMeta {
22+
23+
public static final def DB_URL_FLAG = 'db-url'
24+
public static final def DB_USERNAME_FLAG = 'db-username'
25+
public static final def DB_PASSWORD_FLAG = 'db-password'
26+
public static final def USAGE_FLAG = 'usage'
27+
public static final def HELP_FLAG = 'help'
28+
public static final def SCHEMA_FILE_FLAG = 'schema-file'
29+
30+
public static def cliOption(name) {
31+
"--${name}"
32+
}
33+
34+
public static final def DEFAULT_SCHEMA_FILE_NAME = '/BaseConfigDBSchema.xml'
35+
public static final def DEFAULT_DESTINATION_FILE_NAME = 'SSP_Export_Liquibase_changeset.xml'
36+
37+
public static final def XML_CHANGESET_HEADER = '''<?xml version="1.0" encoding="UTF-8"?>
38+
39+
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
40+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
41+
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
42+
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
43+
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd
44+
http://www.liquibase.org/xml/ns/dbchangelog-ext
45+
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
46+
\n'''
47+
48+
public static final XML_CHANGESET_FOOTER = '''
49+
</databaseChangeLog>
50+
\n'''
51+
52+
public static final USAGE = """
53+
----------------------------------
54+
- Running the program
55+
----------------------------------
56+
57+
The app can be executed in at least three different ways:
58+
59+
1) Gradle run command. This is the easiest way to run the app if you
60+
are tweaking the source code:
61+
62+
%> gradle -q run -PcliArgs="<opts>"
63+
64+
Note that <opts> parsing is naive so spaces in argument values will
65+
not work as expected.
66+
67+
2) Exploded Gradle-assembled application
68+
69+
%> gradle installApp
70+
%> ./build/install/ssp-data-migration/bin/ssp-data-migration <opts>
71+
72+
Opts splitting is handled by your shell in this case, so whitespace
73+
in argument values should work as expected.
74+
75+
Use this execution mechanism if you've received a zip or tar of the
76+
application.
77+
78+
3) Without Gradle. Don't know why you'd want to do this as it is
79+
significantly more verbose and you'll need to do more legwork to
80+
make your JDBC driver visible:
81+
82+
%> groovy -cp \"src/main/groovy/:src/main/resources:/path/to/your/jdbc/driver\" \\
83+
src/main/groovy/org/jasig/ssp/util/migration/Main.groovy \\
84+
<opts>
85+
86+
Output is an XML document on stdout. Redirect as appropriate.
87+
88+
Options all have POSIX "long opt" format, i.e. --option-name=option-value.
89+
The "=" is optional and can be replaced with whitespace. E.g.:
90+
91+
%> gradle -q run -PcliArgs="${cliOption(DB_URL_FLAG)}=jdbc:postgresql://localhost:5432/ssp" > migration.xml
92+
93+
94+
----------------------------------
95+
- Options
96+
----------------------------------
97+
98+
${cliOption(DB_URL_FLAG)} [Required] The full URL to the source database
99+
${cliOption(DB_USERNAME_FLAG)} [Optional] The username for the source database
100+
${cliOption(DB_PASSWORD_FLAG)} [Optional] The password for the source database
101+
${cliOption(SCHEMA_FILE_FLAG)} [Optional] Specify the location for the DB Schema file - the file that specifies what data to export (default: March 2013 tables from last export job)
102+
${cliOption(USAGE_FLAG)}, ${HELP_FLAG} Prints this usage message and performs no processing
103+
"""
104+
105+
public static final def UNIX_TIMESTAMP = /[0-9]{1,4}-[0-9]{1,2}-[0-9]{1,2} [0-9]{1,2}:[0-9]{1,2}:[0-9]{1,2}.[0-9]{1,2}/
106+
107+
}

‎src/main/groovy/org/jasig/ssp/util/migration/MigrationTableObject.groovy

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
/**
2+
* Licensed to Jasig under one or more contributor license
3+
* agreements. See the NOTICE file distributed with this work
4+
* for additional information regarding copyright ownership.
5+
* Jasig licenses this file to you under the Apache License,
6+
* Version 2.0 (the "License"); you may not use this file
7+
* except in compliance with the License. You may obtain a
8+
* copy of the License at:
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on
14+
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
package org.jasig.ssp.util.migration
20+
21+
import groovy.transform.ToString
22+
23+
@ToString
24+
class MigrationTableObject {
25+
def tableName
26+
def columns = [:]
27+
}

‎src/main/groovy/org/jasig/ssp/util/migration/MigrationTableObjectDAO.groovy

+67
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
/**
2+
* Licensed to Jasig under one or more contributor license
3+
* agreements. See the NOTICE file distributed with this work
4+
* for additional information regarding copyright ownership.
5+
* Jasig licenses this file to you under the Apache License,
6+
* Version 2.0 (the "License"); you may not use this file
7+
* except in compliance with the License. You may obtain a
8+
* copy of the License at:
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on
14+
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
package org.jasig.ssp.util.migration
20+
21+
import static MigrationMeta.*
22+
import groovy.sql.Sql
23+
24+
class MigrationTableObjectDAO {
25+
26+
def final postgresDriver = 'org.postgresql.Driver'
27+
def final jtdsDriver = 'net.sourceforge.jtds.jdbc.Driver'
28+
29+
def password = 'sspadmin'
30+
def user = 'sspadmin'
31+
def url
32+
def driver
33+
34+
public MigrationTableObjectDAO(opts) {
35+
36+
if (!(opts[DB_URL_FLAG])) {
37+
throw new IllegalArgumentException("Must specify a database URL")
38+
}
39+
url = opts[DB_URL_FLAG]
40+
41+
if (url.contains('postgresql')) {
42+
driver = postgresDriver
43+
} else if (url.contains('jtds')) {
44+
driver = jtdsDriver
45+
} else {
46+
throw new IllegalArgumentException("Unable to detect target database type. Expect database url to contain either \"postgresql\" or \"jtds\"")
47+
}
48+
49+
user = opts[DB_USERNAME_FLAG]
50+
password = opts[DB_PASSWORD_FLAG]
51+
}
52+
53+
def selectAllFromIteratedTables(def tables) {
54+
def tableObjects = []
55+
def sql = Sql.newInstance(url, user, password, driver)
56+
tables.each { table ->
57+
sql.eachRow("SELECT * FROM ${Sql.expand(table.tableName)}") { result ->
58+
def tableObject = new MigrationTableObject()
59+
tableObject.tableName = table.tableName
60+
tableObject.columns = result.toRowResult()
61+
tableObjects.add(tableObject);
62+
}
63+
}
64+
sql.close()
65+
tableObjects
66+
}
67+
}

‎src/main/groovy/org/jasig/ssp/util/migration/MigrationXMLParser.groovy

+51
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
/**
2+
* Licensed to Jasig under one or more contributor license
3+
* agreements. See the NOTICE file distributed with this work
4+
* for additional information regarding copyright ownership.
5+
* Jasig licenses this file to you under the Apache License,
6+
* Version 2.0 (the "License"); you may not use this file
7+
* except in compliance with the License. You may obtain a
8+
* copy of the License at:
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on
14+
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
package org.jasig.ssp.util.migration
20+
21+
import static org.jasig.ssp.util.migration.MigrationMeta.*
22+
23+
class MigrationXMLParser {
24+
25+
def filePath
26+
27+
public MigrationXMLParser(opts) {
28+
if (opts[SCHEMA_FILE_FLAG]) {
29+
filePath = opts[SCHEMA_FILE_FLAG]
30+
} else {
31+
throw new IllegalArgumentException("Must specify a schema file");
32+
}
33+
}
34+
35+
/** Parses the schema DB Schema XML so the program knows which
36+
* tables to pull data from.
37+
*
38+
* @return List&lt;MigrationTableObject&gt;
39+
*/
40+
public def parseDBSchemaFile() {
41+
def tableObjects = []
42+
43+
def tables = new XmlParser().parse(filePath)
44+
tables.each { table ->
45+
def tableObject = new MigrationTableObject()
46+
tableObject.tableName = table.@tableName
47+
tableObjects.add(tableObject)
48+
}
49+
tableObjects
50+
}
51+
}

‎src/main/groovy/org/jasig/ssp/util/migration/MigrationXMLWriter.groovy

+204
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
/**
2+
* Licensed to Jasig under one or more contributor license
3+
* agreements. See the NOTICE file distributed with this work
4+
* for additional information regarding copyright ownership.
5+
* Jasig licenses this file to you under the Apache License,
6+
* Version 2.0 (the "License"); you may not use this file
7+
* except in compliance with the License. You may obtain a
8+
* copy of the License at:
9+
*
10+
* http://www.apache.org/licenses/LICENSE-2.0
11+
*
12+
* Unless required by applicable law or agreed to in writing,
13+
* software distributed under the License is distributed on
14+
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
* KIND, either express or implied. See the License for the
16+
* specific language governing permissions and limitations
17+
* under the License.
18+
*/
19+
package org.jasig.ssp.util.migration
20+
21+
import static MigrationMeta.*
22+
23+
class MigrationXMLWriter {
24+
25+
def previousTable
26+
def tab = ' '
27+
def changesetCounter = 1
28+
def result
29+
30+
def MigrationXMLWriter(writer, opts) {
31+
result = writer
32+
}
33+
34+
/** Writes the Liquibase XML for the given List&lt;MigrationTableObject&gt;
35+
*
36+
* @param tableObjects a List&lt;MigrationTableObject&gt; of table objects to write the changesets for
37+
* @return result a String containing the liquibase XML
38+
*/
39+
def writeXMLOutput(def tableObjects) {
40+
result << XML_CHANGESET_HEADER
41+
appendDisableMSSQLForeignKey()
42+
43+
tableObjects.each { table ->
44+
previousTable == table.tableName ? changesetCounter++ : (newTableSection(table))
45+
appendTableChangeset(table)
46+
previousTable = table.tableName
47+
}
48+
appendEnablePGForeignKey(previousTable)
49+
50+
result << '\n<!-- ----------------------------------------- -->\n'
51+
result << "<!-- Footer -->\n"
52+
result << '<!-- ----------------------------------------- -->\n\n'
53+
54+
appendEnableMSSQLForeignKey()
55+
result << XML_CHANGESET_FOOTER
56+
}
57+
58+
/** Appends a complete changesets for a MigrationTableObject
59+
*
60+
* @param table a MigrationTableObject to write a changesets for
61+
* @return nothing (appends to class variable)
62+
*/
63+
def appendTableChangeset(def table) {
64+
appendOpenChangeset(table, 'insert')
65+
appendOpenInsert(table)
66+
appendColumns(table)
67+
appendCloseInsert()
68+
appendCloseChangeset()
69+
}
70+
71+
/** Iterates through the columns of a MigrationTableObject and
72+
* calls the write method to generate the liquibase logic to
73+
* inject that data.
74+
*
75+
* @param table the MigrationTableObject that contains the column data
76+
* @return nothing (appends to class variable)
77+
*/
78+
def appendColumns(def table) {
79+
table.columns.each { columnName, columnValue ->
80+
appendColumn(columnName, columnValue)
81+
}
82+
}
83+
84+
/** Writes an individual column changeset with appropriate formatting
85+
* for the data value/type
86+
*
87+
* @param columnName the name of the column to write
88+
* @param columnValue the value of the column to write
89+
* @return nothing (appends to class variable)
90+
*/
91+
def appendColumn(def columnName, def columnValue) {
92+
if (columnValue.toString().contains('<')) {
93+
result << "${tab}<column name=\"${columnName}\"><![CDATA[ \n${columnValue}\n]]>\n${tab}</column>\n"
94+
} else if (columnValue.toString() =~ UNIX_TIMESTAMP) {
95+
result << "${tab}<column name=\"${columnName}\" valueDate=\"${columnValue}\" />\n"
96+
} else {
97+
result << "${tab}<column name=\"${columnName}\" value=\"${columnValue}\" />\n"
98+
}
99+
}
100+
101+
/** Opens a changeset tag for a table insertion
102+
*
103+
* @param table the table the changeset is for
104+
* @param action the action to take with the changeset (e.g. insert, delete, etc.)
105+
* @return
106+
*/
107+
def appendOpenChangeset(def table, def action) {
108+
def change = "${tab}<changeSet id=\"transfer reference data - ${action ?: action} ${table?.tableName} ${changesetCounter}\" author=\"configuration_data_exporter\">\n"
109+
indentTab()
110+
result << change
111+
}
112+
113+
/** Closes a changeset tag, decrements the tab
114+
*
115+
*/
116+
def appendCloseChangeset() {
117+
decrementTab()
118+
result << "${tab}</changeSet>\n\n"
119+
}
120+
121+
/** Opens a liquibase insertion tag
122+
*
123+
* @param table the table object that will be inserted into
124+
* @return
125+
*/
126+
def appendOpenInsert(def table) {
127+
def change = "${tab}<insert tableName=\"${table.tableName}\">\n"
128+
indentTab()
129+
result << change
130+
}
131+
132+
/** Closes an insert tag
133+
*
134+
* @return
135+
*/
136+
def appendCloseInsert() {
137+
decrementTab()
138+
result << "${tab}</insert>\n"
139+
}
140+
141+
/** performs the logic for the liquibase at the point where the table
142+
* being worked on is changed.
143+
*
144+
* @param table
145+
* @return
146+
*/
147+
def newTableSection(def table) {
148+
changesetCounter = 1
149+
appendEnablePGForeignKey(previousTable)
150+
appendTableComment(table)
151+
appendDisablePGForeignKey(table)
152+
appendOpenChangeset(table, 'delete')
153+
appendDeleteTable(table)
154+
appendCloseChangeset()
155+
}
156+
157+
def appendDeleteTable(def table) {
158+
result << "${tab}<sql> DELETE FROM ${table.tableName} </sql>\n"
159+
}
160+
161+
def appendTableComment(def table) {
162+
result << '\n<!-- ----------------------------------------- -->\n'
163+
result << "<!-- Changeset section: ${table.tableName} -->\n"
164+
result << '<!-- ----------------------------------------- -->\n\n'
165+
}
166+
167+
def appendDisableMSSQLForeignKey() {
168+
result << "${tab}<changeSet id=\"transfer reference data - disable foreign key MSSQL\" author=\"configuration_data_exporter\" dbms=\"mssql\">\n"
169+
indentTab()
170+
result << tab + '<sql>execute sp_msforeachtable "ALTER TABLE ? NOCHECK CONSTRAINT all"</sql>\n'
171+
appendCloseChangeset()
172+
}
173+
174+
def appendEnableMSSQLForeignKey() {
175+
result << "${tab}<changeSet id=\"transfer reference data - enable foreign key MSSQL\" author=\"configuration_data_exporter\" dbms=\"mssql\">\n"
176+
indentTab()
177+
result << tab + '<sql>execute sp_msforeachtable "ALTER TABLE ? WITH CHECK CHECK CONSTRAINT all"</sql>\n'
178+
appendCloseChangeset()
179+
}
180+
181+
def appendDisablePGForeignKey(table) {
182+
result << "${tab}<changeSet id=\"transfer reference data - disable ${table.tableName} foreign key Postgres\" author=\"configuration_data_exporter\" dbms=\"postgresql\">\n"
183+
indentTab()
184+
result << "${tab}<sql>ALTER TABLE ${table.tableName} DISABLE TRIGGER ALL</sql>\n"
185+
appendCloseChangeset()
186+
}
187+
188+
def appendEnablePGForeignKey(tableName) {
189+
if (tableName != null) {
190+
result << "${tab}<changeSet id=\"transfer reference data - enable ${tableName} foreign key Postgres\" author=\"configuration_data_exporter\" dbms=\"postgresql\">\n"
191+
indentTab()
192+
result << "${tab}<sql>ALTER TABLE ${tableName} ENABLE TRIGGER ALL</sql>\n"
193+
appendCloseChangeset()
194+
}
195+
}
196+
197+
def indentTab() {
198+
tab += tab
199+
}
200+
201+
def decrementTab() {
202+
tab = tab[0..(tab.length()*-1)]
203+
}
204+
}

‎src/main/resources/BaseConfigDBSchema.xml

+641
Large diffs are not rendered by default.

‎src/main/resources/ConfigDataTableSchema.xsd

+49
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<!--
3+
4+
Licensed to Jasig under one or more contributor license
5+
agreements. See the NOTICE file distributed with this work
6+
for additional information regarding copyright ownership.
7+
Jasig licenses this file to you under the Apache License,
8+
Version 2.0 (the "License"); you may not use this file
9+
except in compliance with the License. You may obtain a
10+
copy of the License at:
11+
12+
http://www.apache.org/licenses/LICENSE-2.0
13+
14+
Unless required by applicable law or agreed to in writing,
15+
software distributed under the License is distributed on
16+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
17+
KIND, either express or implied. See the License for the
18+
specific language governing permissions and limitations
19+
under the License.
20+
21+
-->
22+
<schema xmlns="http://www.w3.org/2001/XMLSchema"
23+
targetNamespace="http://http://www.jasig.org//ConfigDataTableSchema"
24+
xmlns:tns="http://http://www.jasig.org//ConfigDataTableSchema"
25+
elementFormDefault="qualified">
26+
27+
<element name="configData">
28+
<complexType>
29+
<sequence>
30+
<element name="table" minOccurs="1" maxOccurs="unbounded">
31+
<complexType>
32+
<sequence>
33+
<element name="column" minOccurs="1"
34+
maxOccurs="unbounded">
35+
<complexType>
36+
<attribute name="columnName" type="string"
37+
use="required"/>
38+
</complexType>
39+
</element>
40+
</sequence>
41+
<attribute name="tableName" type="string"
42+
use="required"/>
43+
</complexType>
44+
</element>
45+
</sequence>
46+
</complexType>
47+
</element>
48+
49+
</schema>

0 commit comments

Comments
 (0)
Please sign in to comment.