Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Request - Update #186

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion HTTP Request/HTTP Request.step

Large diffs are not rendered by default.

122 changes: 100 additions & 22 deletions HTTP Request/README.md

Large diffs are not rendered by default.

92 changes: 92 additions & 0 deletions HTTP Request/extras/BatchMode_I/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Query post code information in batch mode
The example is calling a REST API using the POST method to enrich UK postal codes with county, council and parish information.<br>
The input table has a column postcode. We group the postal codes from several rows together and submit each group in one call.<br>
The column name from the input table is used in the batch repeat structure and the batch data is added to the HTTP payload.<br>
The required fields from the HTTP result are mapped to columns in the output table.

![](../../img/HTTPRequest_ex6.gif)

---
## Demo recreate
Use the following settings to recreate the above example in SAS Studio.

1. Create new flow job in SAS Studio.
2. Drag table 'zip' on the canvas.
* See [here](#testdata-) to create table 'zip'.
3. ***HTTP Request***
> * Drag ***HTTP Request step*** on canvas and connect with the input table step.
> * Go to tab ***HTTP Request***.
> * Set ***URL*** as below.
> ```
> http://api.postcodes.io/postcodes
> ```
> * Set ***Method*** to *POST*.
> * Fill ***Payload*** text box with below JSON structure. In the JSON structure we use the columns from the input table as parameters.
> ```
> {
> "postcodes" : [@postcode_batch@]
> }
> ```
> * Go to tab ***Batch***.
> * Set *Record group size* to 3.
> * Put below data into text box *Repeated json structure*.<br>
> ```
> "@postcode@"
> ```
> This is the repeated data structure the represents **one** batch record. In this case it is only one field.
> * Put below token variable name into *Batch structure name*.
> ```
> postcode_batch
> ```
> The *token variable name* represents the batch data and is used in the payload in tab *HTTP Request*.
> * Go to tab ***Input Options***.
> * Under ***Headers*** set *Header Lines* to 2.
> * For the first Header Line use default value.<br>
> ```
> "Content-Type"="application/json"
> ```
> * For the second Header Line use below setting.<br>
> ```
> "Accept"="application/json"
> ```
> * Go to tab ***Output Options***.
> * Under ***Output Body - Output Table***<br>
> * Use the below mapping in field *Field Mapping* to copy fields from the URL JSON result structure to the output table.
> ```
> result/0/result/postcode | postcode_1,
> result/0/result/admin_county | county_1,
> result/0/result/admin_district | council_1,
> result/0/result/parish | parish_1,
> result/1/result/postcode | postcode_2,
> result/1/result/admin_county | county_2,
> result/1/result/admin_district | council_2,
> result/1/result/parish | parish_2,
> result/2/result/postcode | postcode_3,
> result/2/result/admin_county | county_3,
> result/2/result/admin_district | council_3,
> result/2/result/parish | parish_3
> ```
> * Add ***Output Port***.
> * Use right mouse click to add output port to the step.
4. ***Run Demo***
* Execute Demo Flow.

### Test Data <a name="testdata-"></a>
Run this code in SAS Studio to create the input data for the demo.

```
data zip;
length postcode $10;
infile cards dlm=",";
input postcode $;
cards;
AL3 8EE
AL4 0RQ
W2 1JU
OX49 5NU
M32 0JG
NE30 1DP
LU6 3DT
;
run;
```
1,694 changes: 1,694 additions & 0 deletions HTTP Request/extras/BatchMode_II/Decision-Flow-Batch.json

Large diffs are not rendered by default.

29 changes: 29 additions & 0 deletions HTTP Request/extras/BatchMode_II/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Query Decision-Flow in batch mode
The example is calling a decision flow from SAS Intelligent Decisioning. The decision flow is running in MAS and is called via its REST API.<br>
The decision flow is looking for Comedy movies with actors 'Adam Sandler', 'Emma Thompson' or 'John Cleese' in it and sets an appropriate flag.<br>
To be able to run the decision flow in batch mode it takes a list of movies in a datagrid as input parameter and returns the datagrid with an additional flag to mark the movies.<br>
In SAS Studio we have a table listing 9000+ movies. We use the HTTP Request Step in batch mode to take slices of 2000 movies per HTTP request. This way we call the decision flow 5 times instead of 9000+ times.


![](../../img/HTTPRequest_ex7.gif)

---

## Run the demo in your environment

To run the demo in your Viya environment follow the steps below:

1. Download files *Decision-Flow-Batch.json* and *n_movies.csv.zip*.
2. Unzip *n_movies.csv.zip*.
3. Import *n_movies.csv* into SAS Viya library *CASUSER*.<br>
If you cannot import into *CASUSER* chose different library.
4. Import Decision-Flow-Batch.json into SAS Viya.<br>
Import through Environment Manager or through Intelligent Decisioning (right burger menu 'Import objects from JSON file' - Viya 2024.06 or later)
5. Go to Intelligent Decisioning:<br>
5.1 Publish Decision-Flow *selectMovies* to MAS with published name: *selectMovies*
6. Go to SAS Studio:<br>
6.1 Open *SAS Content/Public/Demo/HTTP Request/movie/Call Decisionflow – Batch.flw*<br>
6.2 Ensure the step N_MOVIES is pointing at the imported dataset n_movies. If N_MOVIES is not in CASUSER point to the correct location.
7. Run Flow job

![](../../img/HTTPRequest_ex7_install.gif)
Binary file added HTTP Request/extras/BatchMode_II/n_movies.csv.zip
Binary file not shown.
1 change: 1 addition & 0 deletions HTTP Request/extras/CallInternalViyaAPI/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Use the HTTP Request Step to call Viya REST API to update the global variable in
---
## Demo Recreate
Use the following settings to recreate the above example in SAS Studio.

1. Run code section [Test Data](#testdata-) to create a Global Variable in SAS Intelligent Decisioning.
2. Create new flow job in SAS Studio.
3. Step ***Get Global Variable Id***
Expand Down
2 changes: 1 addition & 1 deletion HTTP Request/extras/EnrichDataFlow_GET_I/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Use the following settings to recreate the above example in SAS Studio.
* Execute Demo Flow.

### Test Data <a name="testdata-"></a>
Run this code in SAS Studio to create the input data from the demo.
Run this code in SAS Studio to create the input data for the demo.
```
data work.country;
length country $30;
Expand Down
2 changes: 1 addition & 1 deletion HTTP Request/extras/EnrichDataFlow_GET_II/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Use the following settings to recreate the above example in SAS Studio.
* Execute Demo Flow.

### Test Data <a name="testdata-"></a>
Run this code in SAS Studio to create the input data from the demo.
Run this code in SAS Studio to create the input data for the demo.
```
data work.address;
length address town country $30;
Expand Down
2 changes: 1 addition & 1 deletion HTTP Request/extras/EnrichDataFlow_POST/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Use the following settings to recreate the above example in SAS Studio.
* Execute Demo Flow.

### Test Data <a name="testdata-"></a>
Run this code in SAS Studio to create the input data from the demo.
Run this code in SAS Studio to create the input data for the demo.
```
data work.postcodes;
length po1-po03 $10;
Expand Down
4 changes: 2 additions & 2 deletions HTTP Request/extras/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@ In the sub-folders you find examples how to use the HTTP Request step in deffere
3. [Enrich Data Flow using POST](../extras/EnrichDataFlow_POST)
4. [Use HTTP result to macro and output data to SAS Library](../extras/OutputToSASLib)
5. [Update Global Variable in SAS Intelligent Decisioning using SAS Viya Services](../extras/CallInternalViyaAPI)


6. [Call HTTP request in Batch mode I](../extras/BatchMode_I)
6. [Call HTTP request in Batch mode II](../extras/BatchMode_II)
Binary file added HTTP Request/img/HTTPRequest-Batch-fl.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified HTTP Request/img/HTTPRequest-HTTPRequest-fl.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified HTTP Request/img/HTTPRequest-HTTPRequest-sa.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified HTTP Request/img/HTTPRequest-InputOptions-fl.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified HTTP Request/img/HTTPRequest-OutputOptions-fl.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified HTTP Request/img/HTTPRequest-Settings-fl.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added HTTP Request/img/HTTPRequest_ex6.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added HTTP Request/img/HTTPRequest_ex7.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added HTTP Request/img/HTTPRequest_ex7_install.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.