Merge pull request #125 from Esri/v.next

Merging 1.1 into master and rolling v.next ahead to Pro 1.2 development

README.md

@@ -32,7 +32,7 @@ The ArcGIS Defense and Intelligence Military Features Data repository is a set o
     * If using the Style and Layer Files
 * ArcGIS Runtime 10.2 (or later)
     * If using the Symbol Dictionary Files
-* ArcGIS Professional or ArcGIS Runtime 10.2.4+
+* ArcGIS Pro 1.1 or ArcGIS Runtime 10.3+
     * If using the Stylx Files
 
 ## Instructions

data/mil2525d/README.md

@@ -24,7 +24,7 @@ See the Component Sections for more information:
 
 ## Requirements
 
-* ArcGIS Professional 1.0+
+* ArcGIS Professional 1.1+
 * ArcGIS Runtime 10.3+
 * ArcGIS Desktop (10.1+) - only the .Style Files are compatible with Desktop 10.X
 

data/mil2525d/core_data/gdbs/README.md

@@ -9,62 +9,67 @@ This folder contains the Military Features Core Geodatabase Schema format and an
 
 The following information is intended to succinctly capture the format of the Military Features Schema for other applications that may depend upon this format. 
 
-## Schema Sections
-
-* [Symbol Display Attributes](#symbol-display-attributes)
-* [Modifier (Label) Attributes](#modifier-label-attributes)
-
-## Symbol Display Attributes
+## Graphic Display Amplifiers
 
 The following fields control the display of symbols.
 
-| Attribute Name | Data Type | Range of Values | Default Value/Meaning | Sample Name | Sample Value/Meaning | Explanatory Notes |
-| -------------- | --------- | --------------- | ----------- | -------------------- | ----------------- | ----------------- |
-| identity | long int | 0-9 | 1 (Unknown) | "Friend" | 3 ("friend") | **REQUIRED** |
-| symbolset | short int | 00-99  | (per feature class) | "Air" | 01 ("air") | **REQUIRED** | 
-| entity  | long int | N/A | (per feature class) | "Military (Air) : Fixed-Wing" | "110100"  | **REQUIRED** |
-| context  | short int | 0-2 | 0 (Reality) | "Reality" | 0 ("reality") | Optional |
-| modifier1 | long int | 00-99 | 00 (None) | "Mobility : Air Mobile/Air Assault" | Modifier "01" for Modifier 1 | Optional |
-| modifier2 | long int | 00-99 | 00 (None)  | "Close Range and Support : Casualty Staging" | Modifier "05" for Modifier 2 | Optional |
-| indicator | long int | 0-7 | 0 (None) | Headquarters | Headquarters=2 | Optional (="HQ/TF/FD") |
-| echelon | long int | 0-26 | 0 (None) | Squad  | Squad=12 | Optional |
-| mobility | long int | 0,31-52 (None) | 0 | Rail  | Rail=36 |  Optional |
-| array | long int | 0,61-62 | 0 (None) | Short Towed Array | Short Towed Array=61 | Optional |
-| operationalcondition | long int | 0-5 | 0 (Present) | Planned | Planned=1 | Optional |
-| sidc | string/TEXT | string length(8 or 20) | N/A | "01100110" | SymbolSet:"Air"/"01" + Entity:"100110" | Optional-allows len=20 or len=8(identity=unknown) SIDC format |
+| Attribute Name | 2525 D Field ID | Data Type | Range of Values | Default Value/Meaning | Sample Name | Sample Value/Meaning | Explanatory Notes |
+| -------------- | --------- | --------- | --------------- | ----------- | -------------------- | ----------------- | ----------------- |
+| symbolset | A/E/AC | short int | 00-99  | (per feature class) | "Air" | 01 ("air") | **REQUIRED** | 
+| entity  | A | long int | N/A | (per feature class) | "Military (Air) : Fixed-Wing" | "110100"  | **REQUIRED** |
+| modifier1 | A | long int | 00-99 | 00 (None) | "Mobility : Air Mobile/Air Assault" | Modifier "01" for Modifier 1 | Optional |
+| modifier2 | A | long int | 00-99 | 00 (None)  | "Close Range and Support : Casualty Staging" | Modifier "05" for Modifier 2 | Optional |
+| echelon | B | long int | 0-26 | 0 (None) | Squad  | Squad=12 | Optional |
+| indicator | D/S/AB | long int | 0-7 | 0 (None) | Headquarters | Headquarters=2 | Optional (="HQ/TF/FD") |
+| identity | E | long int | 0-9 | 1 (Unknown) | "Friend" | 3 ("friend") | **REQUIRED** |
+| context  | E | short int | 0-2 | 0 (Reality) | "Reality" | 0 ("reality") | Optional |
+| mobility | R | long int | 0,31-52 (None) | 0 | Rail  | Rail=36 |  Optional |
+| array | AG | long int | 0,61-62 | 0 (None) | Short Towed Array | Short Towed Array=61 | Optional |
+| operationalcondition | AL | long int | 0-5 | 0 (Present) | Planned | Planned=1 | Optional |
+| sidc | | string/TEXT | string length(8 or 20) | N/A | "01100110" | SymbolSet:"Air"/"01" + Entity:"100110" | Optional-allows len=20 or len=8(identity=unknown) SIDC format |
 
-## Modifier (Label) Attributes
+## Text Amplifiers
 
 The following modifier fields are supported in [the current Pro Stylx and Runtime Symbol Dictionary](../stylxfiles). 
 
-To view the Label Rules and associations, view the symbol dictionary table "Label Rules." The following table lists the Modifier definitions from 2525C and 2525D and their corresponding attributes in the symbol dictionary.
+To view the Label Rules and associations, view the symbol dictionary table "Label Rules." The following table lists the Modifier definitions from 2525D and their corresponding attributes in the symbol dictionary.
 
-| Attribute Name | 2525 C Field ID | 2525 D Field ID | Field Title | Notes |
-| ------------- | -------------- | --- | ----------------- | ----- |
-| additionalinformation  | H | H | Additional Information | |
-| additionalinformation2 | H | H | Additional Information 2 | Used in the Convoys and Airspace Coordination Area tactical graphics. |
-| combateffectiveness | K | K | Combat Effectiveness | |
-| commonidentifier | AF | AF | Common Identifier | |
-| credibility | J | J | Evaluation Rating | Credibility rating is second character of Evaluation Rating (J) field. |
-| datetimeexpired | W | W | Date-Time Group (DTG) | Second half of Date-Time Group (DTG) (W) field. |
-| datetimevalid | W | W | Date-Time Group (DTG) | First half of Date-Time Group (DTG) (W) field. |
-| distance | AM | | Distance | |
-| equipmentteardowntime | AE | | Equipment Teardown Time | |
-| higherformation | M | M | Higher Formation | |
-| idmode | P | P | IFF/SIF/AIS | |
-| platformtype | AD | AD | Platform Type | |
-| quantity | C | C | Quantity | |
-| reinforced | F | F | Reinforced or Reduced | |
-| reliability | J | J | Evaluation Rating | Reliability rating is first character of Evaluation Rating (J) field. |
-| sigintmobility | R2 | R2 | SIGINT Mobility Indicator | |
-| signatureequipment | L | L | Signature Equipment | |
-| speed | Z | Z | Speed | |
-| staffcomment | G | G | Staff Comments | |
-| type | V | V | Type | |
-| uniquedesignation | T | T | Unique Designation | |
-| uniquedesignation2 | T | | Unique Designation 2 | Used as an additional field for Fire Support Lines tactical graphics. |
-| x | Y | Y | Location | Longitude in degrees. |
-| y | Y | Y | Location | Latitude in degrees. |
-| z | X | X | Altitude/Depth | |
-| zmax | X | X | Altitude/Depth | Maximum altitude for aviation tactical graphics. |
-| zmin | X | X | Altitude/Depth | Minimum altitude for aviation tactical graphics. |
+| Attribute Name | 2525 D Field ID | Field Title | Notes |
+| ------------- | --- | ----------------- | ----- |
+| quantity | C | Quantity | |
+| reinforced | F | Reinforced or Reduced | |
+| staffcomment | G | Staff Comments | |
+| additionalinformation | H | Additional Information | |
+| additionalinformation2 | H1 | Additional Information 2 | Used in the Convoys and Airspace Coordination Area tactical graphics. |
+| credibility | J | Evaluation Rating | Credibility rating is second character of Evaluation Rating (J) field. |
+| reliability | J1 | Evaluation Rating | Reliability rating is first character of Evaluation Rating (J) field. |
+| combateffectiveness | K | Combat Effectiveness | |
+| signatureequipment | L | Signature Equipment | |
+| higherformation | M | Higher Formation | |
+| hostile | N | Hostile | |
+| idmode | P | IFF/SIF/AIS | |
+| direction | Q | Direction of Movement Indicator | |
+| sigintmobility | R2 | SIGINT Mobility Indicator | |
+| uniquedesignation | T | Unique Designation | |
+| uniquedesignation2 | T1 | Unique Designation 2 | Used as an additional field for Fire Support Lines tactical graphics. |
+| type | V | Type | |
+| datetimevalid | W | Date-Time Group (DTG) | First half of Date-Time Group (DTG) (W) field. |
+| datetimeexpired | W1 | Date-Time Group (DTG) | Second half of Date-Time Group (DTG) (W) field. Many systems use W1 as a trigger to delete symbol from map. |
+| z | X | Altitude/Depth | Maximum altitude for aviation tactical graphics. |
+| z2 | X1 | Altitude/Depth 2 | Minimum altitude for aviation tactical graphics. |
+| x | Y | Location | Longitude in degrees. |
+| x2 | Y1 | Location | Longitude in degrees 2. |
+| y | Y | Location | Latitude in degrees. |
+| y2 | Y1 | Location | Latitude in degrees 2. |
+| speed | Z | Speed | |
+| specialheadquarters | AA | Special C2 Headquarters | |
+| platformtype | AD | Platform Type | |
+| equipmentteardowntime | AE | Equipment Teardown Time | |
+| commonidentifier | AF | Common Identifier | |
+| distance | AM | Distance | |
+| distance2 | AM1 | Distance 2 | |
+| azimuth | AN | Azimuth | |
+| targetdesignator | AP | Target Designator |
+| guardedunit | AQ | Guarded Unit | |
+| specialdesignator | AR | Special Designator | |
+| country | AS | Country Code | |

data/mil2525d/core_data/stylxfiles/README.md

@@ -14,14 +14,14 @@ This folder contains:
     * This auto-generated file is then merged with the .stylx below that is edited/maintained in Pro
     * **This version is intended for end-users**
 * [mil2525d-lines-areas-labels-base-template.stylx](./mil2525d-lines-areas-labels-base-template.stylx) - A separate version of this .stylx that has only the Label Placement Definitions, Point, Line, Polygon, and Text Symbols that are manually created and maintained in ArcGIS Pro
-	* **This version is intended only for use by the Product Engineers of this repo/data**
+    * **This version is intended only for use by the Product Engineers or maintainers of this repo/data**
 
 To use:
 
 [mil2525d.stylx](./mil2525d.stylx)
 
 * In Pro, Add Style to project
-	* **IMPORTANT: THIS FILE SHOULD NEVER BE MANUALLY UPDATED/EDITED IN PRO, IF UPDATING ANYTHING, USE THE OTHER INTENDED VERSION BELOW**
+    * **IMPORTANT: THIS FILE SHOULD NEVER BE MANUALLY UPDATED/EDITED IN PRO, IF UPDATING ANYTHING, USE THE OTHER INTENDED VERSION BELOW**
 * In Runtime 
     * Place the file `mil2525d.stylx` in the folder: `{Runtime Install}\resources\symbols\mil2525d`
     * Requires Versions 10.2.4 or later
@@ -33,3 +33,5 @@ To use:
 
 * In Pro, Add Style to project
 * You may edit/correct the symbols and associated meta data in this file in Pro, update this repo with this file as needed (**for repo Product Engineers or others making edits in Pro only**)
+* Before you edit this file in Pro, please read the [Runtime Compatibility Warning section of the style-utilities README.md](../../utilities/style-utilities/README.md#important-warning-on-runtime-compatibility-before-you-begin)
+* Once you have completed making any desired edits/changes, you will need to merge this file using the steps/process outlined in [merge-stylx-utilities](../../utilities/style-utilities/merge-stylx-utilities)

data/mil2525d/utilities/military-features-utilities/README.md

@@ -3,5 +3,75 @@
 
 ## Purpose
 
-* These utilities are used to create military feature classes, domains, and other feature and test data   
-	* [update-domain-toolbox](./update-domain-toolbox) -  GP Tools for creating geodatabase domains for military feature classes
+* These utilities are used to create military feature classes, domains, and other feature and test data 
+* Individual utilities and steps:
+    * Update Military Features Domains - updates the geodatabase domains from source CSV(.csv) files
+
+## Sections
+
+* [Requirements](#requirements)
+* [Instructions](#instructions)
+
+## Requirements
+
+* ArcGIS Professional 1.0+ (Toolbox requires ArcGIS Pro to open/use)
+
+## Instructions
+
+### Update Military Features Domains
+
+#### Overview
+
+This utility updates the Geodatabase(GDB) domains of the [Military Features template database](../../core_data/gdbs) with the latest source data/values obtained from the [Joint Military Symbology](https://github.com/Esri/joint-military-symbology-xml) repository. This utility should be run periodically as the Joint Military Symbology repository is refined, improved, and updated.
+
+The source data for this utility is a set of CSV files created in the Joint Military Symbology [`name_domains_values folder`](https://github.com/Esri/joint-military-symbology-xml/tree/master/samples/name_domains_values). The domain name is obtained from the CSV file name (with the "Coded_Domain" part removed) and the domain codes and description are obtained from the file contents.
+
+A Geoprocessing (GP) Tool is then run on the source data to add or replace the GDB domains using the source data.
+
+As a final (optional) validation step, once the domain data is imported into the GDB, the domains are then exported and compared to the original source data.
+
+#### Steps
+
+Importing the domain data:
+
+* Obtain the latest set of Military Features source data and utilities
+    * Clone/download this repository to your local machine.
+* Obtain the latest set of `name_domains_values` source CSV files
+    * Clone/download the  [Joint Military Symbology](https://github.com/Esri/joint-military-symbology-xml) repository.
+    * You may also just download the files from the [`name_domains_values folder`](https://github.com/Esri/joint-military-symbology-xml/tree/master/samples/name_domains_values) in this repository
+    * Note: although there are addition sample CSV files in this folder (ex. `*_Sample.csv`), the tools ignore these files using a file filter
+* **REQUIRED WORKAROUND**
+    * There is currently an issue with one of the [source csv files - Coded_Domain_Land_Unit_Entities.csv](https://github.com/Esri/joint-military-symbology-xml/blob/master/samples/name_domains_values/Coded_Domain_Land_Unit_Entities.csv#L193) - there are some **string** entries in this file (containing "XXXX"), but the imported domain codes are expected to be integers
+    * These entries, currently the last four lines of Coded_Domain_Land_Unit_Entities.csv, will need to be manually edited/deleted in NotePad or other **text editor** (you should not use MS Excel because this may alter the format of the csv and data)
+    * See https://github.com/Esri/joint-military-symbology-xml/issues/201 for the full description of this issue and workaround
+* Run ArcGIS Pro
+* Navigate to the local location of the  [update-domain-toolbox GeoProcessing Toolbox](./update-domain-toolbox)
+    * The toolbox should look similar to this
+![Image of Update Domain Toolbox](./screenshots/Toolbox.JPG)
+* Run the *Import or Replace All Domains (Military Features)* GP Tool
+    * As the `Input Folder` select the `joint-military-symbology-xml/tree/master/samples/name_domains_values` folder
+    * As the `Target Geodatabase` select the desired Military Features template geodatabase (usually the one [obtained from here](../../core_data/gdbs)) 
+        * IMPORTANT: The *Table To Domain* operation requires an exclusive schema lock on the geodatabase - therefore:
+        * You should **not** have this geodatabase open elsewhere (for example, added to the current map) while performing this operation.
+        * You must have full editing privileges (Update, Delete, etc.) to any feature class using this domain (mainly an issue if using SDE)
+    * The GP Tool parameters will look similar to the following
+![Image of Import Domains](./screenshots/ScreenShot.JPG)
+* When the tool runs successfully, open the geodatabase in design mode and verify that the domains have been updated with the new source data
+
+Verifying the domain data updates (*Recommended/Optional*):
+
+* After running the *Importing the domain data* steps above
+* Run ArcGIS Pro
+* Navigate to the local location of the  [update-domain-toolbox GeoProcessing Toolbox](./update-domain-toolbox)
+* Run the *Export GDB Domains to Folder* GP Tool
+    * As the `Input Workspace` select the Military Features template geodatabase updated while performing the *Importing the domain data* steps above
+    * As the `Output Folder` select an empty folder
+    * This tool will export all Geodatabase domains to this folder
+![Image of Export Domains](./screenshots/ScreenShot2.JPG)
+* When the tool runs successfully, open the `Output Folder` and verify that the folder contains one CSV file for each domain stored in the `Input Workspace`
+* Using a Diff Utility (such as [WinMerge](http://winmerge.org/)) compare the folder of exported domains to the folder containing the original/source set of `name_domains_values` source CSV files to verify that the exported CSVs match the inported CSVs
+    * Note: a Diff Utility may notice some slight differences, for example
+        * Domains included in the Geodatabase that are not included in the source data
+        * Leading zeroes in the source data imports that are not reflected in the export, for example
+
+ ![Image of Domains Diff](./screenshots/DomainDiff.JPG)

data/mil2525d/utilities/military-features-utilities/update-domain-toolbox/scripts/ExportGDBDomainsToFolder.py

@@ -0,0 +1,73 @@
+# ----------------------------------------------------------------------------------
+# Copyright 2015 Esri
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ----------------------------------------------------------------------------------
+# ExportGDBDomainsToFolder.py
+# Description: Exports all domains from a Military Features GDB
+# Requirements: ArcGIS Desktop/Pro
+# ----------------------------------------------------------------------------------
+import csv
+import os
+import arcpy
+
+def exportDomains():
+	gdb    = arcpy.GetParameter(0)
+	folder = arcpy.GetParameter(1)
+
+	if (gdb == '') or (gdb is None):
+		arcpy.AddError('Input GDB not provided')
+		return
+
+	if (folder == '') or (folder is None):
+		arcpy.AddError('Output folder not provided')
+		return
+
+	domains = arcpy.da.ListDomains(gdb)
+
+	for domain in domains:
+		domainName  = domain.name
+		csvFileName = 'Coded_Domain_' + domainName + '.csv'
+		if domain.domainType == 'CodedValue':
+			arcpy.AddMessage('Exporting Domain: ' + domainName + ' to CSV: ' + csvFileName)
+
+			codedValues = domain.codedValues
+
+			# Create a csv file and writer to export the domains
+			# we do this versus the DomainToTable snippet below* 
+			# so we can make the format closely match the format of the exported tables
+			# at: https://github.com/Esri/joint-military-symbology-xml/tree/master/samples/name_domains_values
+			# (so we can then compare the 2 outputs with a "diff" utility)
+			csvFullFileName = os.path.join(str(folder), csvFileName)
+			csvFile = open(csvFullFileName, 'w', newline='')
+			writer = csv.writer(csvFile, delimiter=',')
+			writer.writerow(["Name", "Value"])  # make header row
+
+			# sort the codes/keys so things are exported in order
+			sortedCodedValues = sorted(codedValues.keys())
+
+			# write each line to the csv in the [description, value] format
+			for val in sortedCodedValues:
+				writer.writerow([codedValues[val], val])
+
+	# *DomainToTable snippet
+	# NOTE a much simpler version of the above if we don't care above the order/format of the export table
+	# desc = arcpy.Describe(gdb)
+	# domains = desc.domains
+	# for domain in domains:
+		# csv = 'Coded_Values_' + str(domain) + '.csv'
+		# arcpy.AddMessage('Exporting: ' + str(domain) + ' to CSV: ' + csv)
+		# table = os.path.join(str(folder), csv)
+		# arcpy.DomainToTable_management(gdb, domain, table, 'Value','Name', '#')
+
+if __name__ == '__main__':
+	exportDomains()