Skip to content

Commit

Permalink
Merge pull request #86 from qsnyder/qms-edits
Browse files Browse the repository at this point in the history
updated parsing xml, json to use aci instead of cmx
  • Loading branch information
agentlecisco authored Sep 11, 2020
2 parents b6cba37 + cd39f6d commit b2374ca
Show file tree
Hide file tree
Showing 50 changed files with 414 additions and 286 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
.DS_Store
venv/
2 changes: 1 addition & 1 deletion labs/coding-101-rest-basics-ga/1.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,6 @@ A good starting point to understand any API is to look at the documentation for
1. Select the latest API version.
1. Keep the DNA Center API Reference documentation open in a separate tab. This page provides a useful reference while you work on this lab.

### Next step
**Next step:**

Proceed to Step 2: So what is a REST web service?
2 changes: 1 addition & 1 deletion labs/coding-101-rest-basics-ga/2.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,6 @@ In this example, you request the list of hosts, and that information is returned

* [XML](https://www.w3schools.com/xml/xml_whatis.asp) - XML, or eXtensible Markup Language, a language designed to store and transport data.

### Next step
**Next step:**

Proceed to Step 3: What do I need to know to make a request?
2 changes: 1 addition & 1 deletion labs/coding-101-rest-basics-ga/3.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,6 @@ Study the Ticket API in DNA Center:

* Look at the other network devices endpoints in the API Reference Guide. How do they differ from one another?

### Next step
**Next step:**

Proceed to Step 4: Construct a REST call.
2 changes: 1 addition & 1 deletion labs/coding-101-rest-basics-ga/4.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,6 @@ To get the token your request will look like this:
* Provide the username and password for logging into the DNA Center in JSON format.


### Next step
**Next step:**

Proceed to Step 5: Make a REST API call.
2 changes: 1 addition & 1 deletion labs/coding-101-rest-basics-ga/5.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,6 @@ On OS X, to use Python 3.4, you may need to accept the SSL certificate before ca

#### Congratulations! You made a first REST API call to DNA Center!

### Next step
**Next step:**

Proceed to Step 6: Use a token to make REST API calls.
2 changes: 1 addition & 1 deletion labs/coding-101-rest-basics-ga/6.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,6 @@ To get the entire list of hosts, your request contains these items:

![](assets/images/HostRequest.png)

### Next step
**Next step:**

Proceed to Step 7: Get the network hosts.
2 changes: 1 addition & 1 deletion labs/coding-101-rest-basics-ga/7.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,6 @@

#### Congratulations! You made your first REST API call using the DNA Center Service Ticket!

### Next step
**Next step:**

Proceed to Step 8: Get the network devices.
6 changes: 5 additions & 1 deletion labs/coding-102-rest-python-ga/1.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ In this Learning Lab, you learn the basics of how to call a REST API in Python.
## Prerequisites

**Background**

* Complete the [Coding Fundamentals](https://developer.cisco.com/learning/modules/programming-fundamentals) and [REST API Fundamentals](https://developer.cisco.com/learning/modules/rest-api-fundamentals) Learning Lab modules if you are unfamiliar with Python and retrieving results from a RESTful service and the [Parsing XML using Python Lab](lab/coding-201-parsing-xml/step/1 "Parsing XML using Python Lab") for a similar approach to retrieving data using XML.

* You should also have a basic familiarity with JSON. Otherwise, consider visiting the [W3Schools JSON Tutorial](https://www.w3schools.com/js/js_json_intro.asp "W3Schools JSON Tutorial") to get a firm base to build upon.
* Complete the [Coding 101 Rest Basics Learning Lab](/lab/coding-101-rest-basics-ga/step/1) before you start this lab.

**Access to DNA Center**
Expand Down Expand Up @@ -65,6 +69,6 @@ A good starting point to understand any API is to look at the documentation for
1. Select the latest API version.
1. Keep the DNA Center API Reference documentation open in a separate tab. This page provides a useful reference while you work on this lab.

### Next step
**Next step:**

Proceed to Step 2: Make your first REST call from Python.
2 changes: 1 addition & 1 deletion labs/coding-102-rest-python-ga/2.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,6 @@ This program displays the raw JSON from the token data.

In the next few sections, you look at how to use the token when making other REST API calls and how to parse the JSON that is returned in the response.

### Next step
**Next step:**

Proceed to Step 2: Make your first REST call from Python.
2 changes: 1 addition & 1 deletion labs/coding-102-rest-python-ga/3.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,6 @@ To run this code sample:

Next, you learn how to get Network Devices and make your source code easier to read.

### Next step
**Next step:**

Proceed to Step 3: What do I need to know to make a request?
2 changes: 1 addition & 1 deletion labs/coding-102-rest-python-ga/4.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,6 @@ To run this code sample:

Next, you learn how to build a network topology.

### Next step
**Next step:**

Proceed to Step 5: Build network topology.
2 changes: 1 addition & 1 deletion labs/coding-102-rest-python-ga/5.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,6 @@ To run this code sample:

Next, you learn how to use the NeXt UI toolkit and Flask to build a network topology and display it graphically.

### Next step
**Next step:**

Proceed to Step 6: Build network topology and graphical display.
82 changes: 54 additions & 28 deletions labs/coding-201-parsing-xml/1.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# Coding 201: Parsing XML using Python
# Parsing XML using Python

In this Learning Lab, you learn the basics of parsing XML content using Python. The example runs a query on [CMX Mobility Services](https://developer.cisco.com/site/cmx-mobility-services/ "CMX Mobility Services") data for available access points and put their names into a simple list for display. To learn more about CMX, review the [Mobility Services Engine (MSE) Learning Lab](lab/cmx/step/1).
In this Learning Lab, you learn the basics of parsing XML content using Python. The example runs a query on the [ACI Always-On Sandbox](https://devnetsandbox.cisco.com/RM/Diagram/Index/5a229a7c-95d5-4cfd-a651-5ee9bc1b30e2?diagramType=Topology "ACI Always-On Sandbox") in order to request data for available configuration on the fabric. We will then parse this returned data and print it to the screen for human readability. To learn more about ACI and its API, you can review the DevNet [learning track for ACI programmability](https://developer.cisco.com/learning/tracks/aci-programmability), which covers a variety of different ways to interact with ACI programmatically. ACI is used as a basis for these labs as the managed object model within ACI allows us to query the same object or class within the fabric and return both XML and JSON data structures, simply based on changing the file-type within the REST URL.

## Objectives

* Understand the basics of reading and parsing XML content from HTTP using Python.
* Evaluate data returned from the CMX Mobility Service API.
* Evaluate data returned from the ACI Always-On Sandbox
* Use Python to extract only the XML data you want, using Minidom.

## Prerequisites

Go through the [Coding 101 lab](lab/coding-101-rest-basics-ga/step/1 "Coding 101 Lab") if you are unfamiliar with Python and retrieving results from a RESTful service.
* Complete the [Coding Fundamentals](https://developer.cisco.com/learning/modules/programming-fundamentals) and [REST API Fundamentals](https://developer.cisco.com/learning/modules/rest-api-fundamentals) Learning Lab modules if you are unfamiliar with Python and retrieving results from a RESTful service and the [Parsing XML using Python Lab](lab/coding-201-parsing-xml/step/1 "Parsing XML using Python Lab") for a similar approach to retrieving data using XML.

You should be familiar with XML. Otherwise, consider visiting the [W3Schools XML Tutorial](https://www.w3schools.com/xml "W3Schools XML Tutorial") to get a firm base to build upon.
* You should also have a basic familiarity with JSON. Otherwise, consider visiting the [W3Schools JSON Tutorial](https://www.w3schools.com/js/js_json_intro.asp "W3Schools JSON Tutorial") to get a firm base to build upon.

For this lab, use Python 3.4+. If you are on a DevNet Zone station, the correct version of Python should already be installed. If you are following this lab online, check the version by entering the following command in a terminal or command prompt window:
```
Expand All @@ -21,29 +21,55 @@ python --version

## Example: Make an HTTP REST call with Python

> **Note**: The example uses a CMX MSE Server, where you would need to have access to one and substitute here.
> **Note**: This lab leverages the DevNet Always-On ACI Sandbox, however you can substitute this with a local APIC if you have one available. The sets of sample code contained within this lab will not perform any write or configuration actions against the fabric; they will only request currently configured data.
To get started, this walk-through shows creating a simple Python script that sends an HTTP request to a CMX MSE Server.
To get started, this walk-through shows creating a simple Python script that sends an HTTP request to the ACI sandbox.

Example `get-ap-xml.py` file:
```
from urllib.request import Request, urlopen
req = Request('https://<ServerURLorIPAddress>/api/contextaware/v1/maps/info/DevNetCampus/DevNetBuilding/DevNetZone')
req.add_header('Authorization', 'Basic bGVhcm5pbmc6bGVhcm5pbmc=')
response = urlopen(req)
responseString = response.read().decode("utf-8")
print(responseString)
response.close()
Example `get-tenants-xml.py` file:
``` python
import requests
import xml.dom.minidom
# We need to import the JSON library just to handle our request to the APIC for login
import json


# We need to log in to the APIC and gather a token, before we can access any data
# Let's construct a request with a body

# We'll need to disable certificate warnings
requests.packages.urllib3.disable_warnings()

# We need to have a body of data consisting of a username and password to gather a cookie from APIC
encoded_body = json.dumps({
"aaaUser": {
"attributes": {
"name": "admin",
"pwd": "ciscopsdt"
}
}
})

# Now lets make the request and store the data
resp = requests.post("https://sandboxapicdc.cisco.com/api/aaaLogin.json", data=encoded_body, verify=False)

# This stores the received APIC-cookie from the login as a value to be used in subsequent REST calls
header = {"Cookie": "APIC-cookie=" + resp.cookies["APIC-cookie"]}

# Now we make a call towards the tenant class on the ACI fabric with the proper header value set.
# We leverage the .xml ending to receive the data back as XML. We're adding health and faults to the printout to ensure that we get levels of data back from the APIC
tenants = requests.get("https://sandboxapicdc.cisco.com/api/node/class/fvTenant.xml?rsp-subtree-include=health,faults", headers=header, verify=False)

# Requests stores the text of the response in the .text attribute. Lets print it to see raw XML
print(tenants.text)
```

This snippet shows:
- Importing the required `Request` and `urlopen` libraries.
- Constructing a request to the CMX URI.
- Adding basic authentication to your Request.
- Opening the request and getting a response back from the CMX URI.
- Parsing the response as a string and printing it to the console.
There's a lot to unpack here, but most of it is related to the idiosyncrasies of ACI and the required authentication. What is important to focus on is that we have:
- imported the `requests` library to easily interact with REST APIs
- constructed a call to authenticate against the APIC and store the authentication token as a cookie to be used later
- created a request to ask for all configured tenants in the fabric using the stored authentication cookie
- accessed the response from the query and printed the raw XML output to the console/terminal

To download or review the current code, you can get it from GitHub <a href="https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding201-parsing-xml/get-ap-xml-1.py" target="_blank">here</a>.
To download or review the current code, you can get it from GitHub <a href="https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding201-parsing-xml/get-tenants-xml-1.py" target="_blank">here</a>.

If you are using Windows, enter the following command into the command prompt window to move to your computer:
```
Expand All @@ -55,19 +81,19 @@ cd ~/Desktop
```
In the command prompt window, enter the following command:
```
python get-ap-xml.py
python get-tenants-xml.py
```

> **Note**: On OS X, to use Python 3.4, you may need to enter `python3.4 get-ap-xml.py`. If you get errors, verify the version of python and double-check the code.
> **Note**: Ensure that the Python version in use is greater than 3.4. If you get errors, verify the version of Python and double-check the code.
When you run the Python script, the terminal displays a screen full of XML data.

![](assets/images/xml-output.png)
![](assets/images/xml-output-new.png)

----------

By default, CMX returns the data in XML. For this first step, this is what you want. In the next step, you will clean up this data and display something more specific.
By using the `.xml` ending to our query, ACI returned the tenant data back to us in XML. For the purposes of this lab, this is exactly what we want. In the next step, you will clean up this data and display something more specific.

### Next step
**Next step:**

Proceed to Step 2: Understand the returned XML data.
60 changes: 25 additions & 35 deletions labs/coding-201-parsing-xml/2.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,64 +6,54 @@ Before parsing the returned XML, take a moment to understand the structure and c

The large XML result that the script returns is a string. Thankfully, Python comes with a library that provides a Minimal DOM implementation called *Minidom*. Mindom can "pretty print" XML in a more human-readable format.

1. To use this functionality, modify `get-ap-xml.py` and insert this import statement on the second line:
```
1. To use this functionality, modify `get-tenants-xml.py` and insert this import statement on the second line, which imports the Minidom library:
```python
import xml.dom.minidom
```
This line of code imports the Minidom library.
<br/>
<br/>
2. Replace the `print(responseString)` line with the following:
```
dom = xml.dom.minidom.parseString(responseString)
2. Replace the `print(tenants.text)` line with the following:
```python
dom = xml.dom.minidom.parseString(tenants.text)
xml = dom.toprettyxml()
print(xml)
```
This snippet:
- Parsing the Response as Document Object Model (DOM)
- Calling a method off of Minidom that returns a String as formatted XML
- Printing the formatted XML
<br/>
<br/>
3. Save the `get-ap-xml.py` file. To download or review the current code, you can get it from GitHub <a href="https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding201-parsing-xml/get-ap-xml-2.py" target="_blank">here</a>.
- parses `response.text` as Document Object Model (DOM)
- calls a method off of Minidom that returns a string as formatted XML
- printing the formatted XML
3. Save the `get-tenants-xml.py` file. To download or review the current code, you can get it from GitHub <a href="https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding201-parsing-xml/get-tenants-xml-2.py" target="_blank">here</a>.

4. Run your `get-ap-xml.py` file using Python:
4. Run your `get-tenants-xml.py` file using Python:
```
python get-ap-xml.py
python get-tenants-xml.py
```
Note: On OS X, to use Python 3.4, you may need to enter `python3.4 get-ap-xml.py`. If you get errors, verify the version of python and double-check the code.
<br/>
<br/>
> **Note**: Ensure that the Python version in use is greater than 3.4. If you get errors, verify the version of Python and double-check the code.
The response is still a formidable amount of text, but you can see some of the structure of the XML document itself.

![](assets/images/xml-output-pretty.png)
![](assets/images/xml-output-pretty-new.png)

----------

A key point to notice is the hierarchical structure of the returned data. Notably, the XML element with the tag name `Floor` has sub-objects which can have objects. For this exercise, you are interested in the collection of Access Points for this floor and, in particular, the name, Ethernet MAC Address, and IP Address of each Access Point. In the information returned, you are interested in the `name`, `ethMacAddress`, and `ipAddress` attributes of elements that have an `sAccessPoint` tag.
A key point to notice is the hierarchical structure of the returned data. Notably, the XML element with the tag name `fvTenant` has sub-objects which can have objects (depending on how we filter our query for data against the APIC). For this exercise, we are interested in the returned tenants (`fvTenant`), their names, and their current health scores. Because the `healthInst` object is a sub-object of the `fvTenant`, we have to move between different levels of the returned XML to get the information we want, in thie case the `name` inside of `fvTenant` and `cur` inside of `healthInst`.

Structurally, the XML elements have the following order:

&lt;Floor&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&lt;Dimension /&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&lt;Image /&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&lt;AccessPoint&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;MapCoordinate /&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ApInterface /&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;ApInterface /&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&lt;/AccessPoint&gt;<br/>
&lt;fvTenant&gt;<br/>
&nbsp;&nbsp;&nbsp;&nbsp;&lt;healthInst/&gt;<br/>
&lt;/fvTenant&gt;<br/>

It may not look interesting, but inside of each of the tags, we can see that multiple values are stored for different properties of that element.

In the description shown, the elements tagged as `AccessPoint` have sub-elements of `MapCoordinate` and `ApInterface`. The `AccessPoint` element has XML attributes that can tell you the name of the Access Point and its MAC and IP addresses. A full `AccessPoint` element from the data can look like this:
In the description shown, the elements tagged as `fvTenant` have sub-elements of `dn` and `modTS` and `name` (there are more, but they will likely be blank), which detail the "Distinguished Name" of the object, its modification time, and its "Relative Name (name)" A full `fvTenant` element from the data can look like this (you'll see the `healthInst` tag nested inside of the `fvTenant`):
```
<AccessPoint apMode="LOCAL" ethMacAddress="00:2b:01:00:05:f0" ipAddress="10.10.20.244" name="T1-4" numOfSlots="2" radioMacAddress="00:2b:01:00:05:00">
<fvTenant annotation="" childAction="" descr="" dn="uni/tn-infra" extMngdBy="" lcOwn="local" modTs="2020-04-19T19:35:27.958+00:00" monPolDn="uni/tn-common/monepg-default" name="infra" nameAlias="" ownerKey="" ownerTag="" status="" uid="0">
<healthInst childAction="" chng="0" cur="100" maxSev="cleared" modTs="never" prev="100" rn="health" status="" twScore="100" updTs="2020-04-22T19:27:33.002+00:00"/>
</fvTenant>
```
Where the tag is "AccessPoint" and there are six attributes: `apMode`, `ethMacAddress`, `ipAddress`, `name`, `numOfSlots`, and `radioMacAddress`.

For a complete reference about map information from CMX, review the online documentation at [https://developer.cisco.com/site/cmx-mobility-services/documents/api-reference-manual/#maps-api](https://developer.cisco.com/site/cmx-mobility-services/documents/api-reference-manual/#maps-api "Online Documentation for CMX Maps API")
Where the tag is "fvTenant" and the attributes follow after the tag. You can also see the health tag and its subsequent attributes.

Now, you have a better understanding of what the structure looks like. Next, you'll parse the data.

### Next step
**Next step:**

Proceed to Step 3: Get XML elements in Python.
Loading

0 comments on commit b2374ca

Please sign in to comment.