Skip to content

Commit dae0483

Browse files
author
swagfinger
committed
update notes
1 parent a8da465 commit dae0483

File tree

8 files changed

+1880
-4
lines changed

8 files changed

+1880
-4
lines changed

‎03.postgreSQL.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -530,5 +530,37 @@ calling the down migration will revert one migration file at a time.
530530
###### <div style="text-align:right">[table of contents](#table-of-contents)</div>
531531

532532
## Data Migration
533+
if you have to update a table's structure,
534+
535+
STEPS:
536+
1. first add the new column (schema migration) allow DEFAULT null values
537+
538+
// time can pass between these steps
539+
540+
2. deploy new API that will write data to both old columns (eg. lat, lng) and new column eg. loc (x, y)
541+
newer posts will have data in all the columns (lat, lng, loc)
542+
543+
3. copy data over (data migration) (eg. (lat, lng) -> loc
544+
every loc value with null value, copy over (lat, lng) to loc
545+
546+
4. update code to write ONLY to loc column
547+
548+
5. drop old column(s) (schema migration) (eg. lat, lng)
549+
550+
DO NOT MIX SCHEMA MIGRATIONS and DATA MIGRATIONS together
551+
ie. use 3 separate migration files instead of one.
552+
553+
##### requirement - execute inside a transaction
554+
You do not want to have a migration in a half executed state.
555+
it is common to execute migration inside a transaction.
556+
557+
if an error occurs, you want to rollback the entire transaction and the entire thing gets canceled and
558+
no changes happen on the database.
559+
560+
because copying data within a database can be a time consuming process;
561+
when copying, postgres is makes a copy of the table at a certain point in time ('on the side') and once complete, a commit is made.. during this time the api server can still be creating requests and saving to the database without the new data being added to the migration copy 'on the side', they are not included as part of the transaction.
562+
the later data is lost and gets 'null' values.
563+
564+
533565

534566
---

‎04.express-server-api.md

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
2+
# NOTES: Express server API
3+
4+
---
5+
6+
## Table of contents
7+
8+
1. [add posts table](#schema-migration-1) - schema migration 1
9+
10+
2. [create web server](#creating-express-web-server)
11+
12+
3. [perform a schema migration](#schema-migration-2) - schema migration 2
13+
14+
---
15+
###### <div style="text-align:right">[table of contents](#table-of-contents)</div>
16+
17+
## schema migration 1
18+
'socialnetwork' is the name of the database.
19+
20+
* create web app / add posts table
21+
* add column loc
22+
* deploy new version of API that will write values to both lat/lng and loc
23+
* copy lat/lng to loc
24+
* update code to only write to loc column
25+
* drop columns lat/lng
26+
27+
28+
29+
//examples/01-express-api
30+
```js
31+
//package.js
32+
"scripts": {
33+
"migrate":"node-pg-migrate"
34+
}
35+
```
36+
37+
```cmd
38+
npm run migrate create add posts table
39+
```
40+
41+
<!-- inside the migration file we just created -->
42+
```js
43+
exports.shorthands = undefined;
44+
45+
exports.up = pgm =>{
46+
pgm.sql(`CREATE TABLE posts (
47+
id SERIAL PRIMARY KEY,
48+
url VARCHAR(300),
49+
lat NUMERIC,
50+
lng NUMERIC
51+
);
52+
`);
53+
};
54+
55+
exports.down = pgm =>{
56+
pgm.sql(`
57+
DROP TABLE posts;
58+
`);
59+
};
60+
```
61+
62+
```cmd
63+
set DATABASE_URL=postgres://USERNAME:PASSWORD@localhost:5432/socialnetwork&&npm run migrate up
64+
```
65+
66+
---
67+
###### <div style="text-align:right">[table of contents](#table-of-contents)</div>
68+
69+
## creating express web server
70+
Directions:
71+
72+
1) At your terminal in your project directory, run npm install express pg
73+
2) Download the index.js file attached to this lecture and place it in your project directory
74+
3) Open up the index.js file and update the user and password on lines 8 and 9 to the username and password that you use to connect to your PG database. If you are on macOS, the user is your username and the password is probably an empty string. If you are on windows, the user is probably postgres and the password is probably whatever you set it to during the initial Postgres install.
75+
4) Save the file
76+
5) At the terminal, run node index.js
77+
6) Open your browser and navigate to http://localhost:3005/posts
78+
7) Try to create a new post. You should see it appear on the table
79+
80+
81+
* npmjs.com/package/node-pg-migrate
82+
83+
examples/01-express-api/
84+
85+
```cmd
86+
npm install express node-pg-migrate pg
87+
88+
```
89+
90+
91+
to run index.js
92+
```
93+
node index.js
94+
```
95+
96+
```browser
97+
localhost:3005/posts
98+
```
99+
100+
```js
101+
// index.js
102+
const express = require('express');
103+
const pg = required('pg');
104+
105+
// connection details
106+
const pool = new pg.Pool({
107+
host:'localhost',
108+
port: 5432,
109+
database: 'socialnetwork',
110+
111+
// same details as on migrate command
112+
user: '' //windows is probably 'postgres'
113+
password: ''
114+
});
115+
116+
//running a query
117+
//pool.query('SELECT 1+1;').then((res)=> console.log(res));
118+
119+
const app = express();
120+
121+
122+
//allow us to received foreign submissions from a browser
123+
app.use(express.urlencoded({extended:true}));
124+
125+
//route handlers
126+
//show posts
127+
app.get('/posts', async (req, res)=>{
128+
const {rows} = await pool.query(`
129+
SELECT * FROM posts;
130+
`);
131+
132+
res.send(`
133+
<table>
134+
<thead>
135+
<tr>
136+
<th>id</th>
137+
<th>lng</th>
138+
<th>lat</th>
139+
</tr>
140+
</thead>
141+
<tbody>
142+
${rows.map(row=>{
143+
return `
144+
<tr>
145+
<td>${row.id}</td>
146+
<td>${row.lng}</td>
147+
<td>${row.lat}</td>
148+
149+
</tr>
150+
`
151+
}).join('')}
152+
</tbody>
153+
</table>
154+
<form method="POST">
155+
<h3>create post</h3>
156+
<div>
157+
<label>Lng</label>
158+
<input name="lng"/>
159+
</div>
160+
<div>
161+
<label>Lng</label>
162+
<input name="lat"/>
163+
</div>
164+
<button type="submit">create</button>
165+
</form>
166+
`);
167+
});
168+
169+
app.post('/posts', async (req, res)=>{
170+
console.log(req.body);
171+
//console.log('lng: ', lng, 'lat: ', lat);
172+
const {lng, lat} = req.body;
173+
await pool.query('INSERT INTO posts (lat, lng) VALUES ($1, $2);', [lat, lng]);
174+
res.redirect('posts');
175+
});
176+
177+
app.listen(3005, ()=>{
178+
console.log('listening on port 3005');
179+
});
180+
181+
```
182+
183+
---
184+
###### <div style="text-align:right">[table of contents](#table-of-contents)</div>
185+
186+
## Schema migration 2
187+
adding the loc column
188+
189+
190+
---

‎README.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,6 @@
11
# NOTES: SQL + PostgreSQL
22

3-
## Source
4-
* Stephen Grider - SQL and PostgreSQL The complete developers guide
5-
completion date: 2021-10-16
63

7-
## skills gained?
84
* SQL
95
* database design
106
* schema designer
@@ -16,4 +12,10 @@ completion date: 2021-10-16
1612
+ [1. SQL](https://github.com/swagfinger/sql_postgreSQL/blob/master/01.SQL.md)
1713
+ [2. database design](https://github.com/swagfinger/sql_postgreSQL/blob/master/02.database_design.md)
1814
+ [3. postgreSQL](https://github.com/swagfinger/sql_postgreSQL/blob/master/03.postgreSQL.md)
15+
+ [4. Express server API](https://github.com/swagfinger/sql_postgreSQL/blob/master/04.express-server-api.md)
16+
1917

18+
---
19+
20+
## Source
21+
* Stephen Grider - SQL and PostgreSQL The complete developers guide

‎examples/01-express-api/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
node_modules/

‎examples/01-express-api/index.js

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
const express = require('express');
2+
const pg = require('pg');
3+
4+
// connection details
5+
const pool = new pg.Pool({
6+
host:'localhost',
7+
port: 5432,
8+
database: 'socialnetwork',
9+
10+
// same details as on migrate command
11+
user: 'postgres', //windows is probably 'postgres'
12+
password: 'Helloworld'
13+
});
14+
15+
//running a query
16+
//pool.query('SELECT 1+1;').then((res)=> console.log("test here: ", res));
17+
18+
const app = express();
19+
20+
21+
//allow us to received foreign submissions from a browser
22+
app.use(express.urlencoded({extended:true}));
23+
24+
//route handlers
25+
//show posts
26+
app.get('/posts', async (req, res)=>{
27+
const {rows} = await pool.query(`
28+
SELECT * FROM posts;
29+
`);
30+
31+
res.send(`
32+
<table>
33+
<thead>
34+
<tr>
35+
<th>id</th>
36+
<th>lat</th>
37+
<th>lng</th>
38+
</tr>
39+
</thead>
40+
<tbody>
41+
${rows.map(row=>{
42+
return `
43+
<tr>
44+
<td>${row.id}</td>
45+
<td>${row.lat}</td>
46+
<td>${row.lng}</td>
47+
</tr>
48+
`
49+
}).join('')}
50+
</tbody>
51+
</table>
52+
<form method="POST">
53+
<h3>create post</h3>
54+
<div>
55+
<label>Lng</label>
56+
<input name="lng"/>
57+
</div>
58+
<div>
59+
<label>Lng</label>
60+
<input name="lat"/>
61+
</div>
62+
<button type="submit">create</button>
63+
</form>
64+
`);
65+
});
66+
67+
app.post('/posts', async (req, res)=>{
68+
const {lng, lat} = req.body;
69+
await pool.query('INSERT INTO posts (lat, lng) VALUES ($1, $2);', [lat, lng]);
70+
res.redirect('posts');
71+
});
72+
73+
app.listen(3005, ()=>{
74+
console.log('listening on port 3005');
75+
});

‎examples/01-express-api/migrations/1635452656158_add-posts-table.js

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
/* eslint-disable camelcase */
2+
3+
exports.shorthands = undefined;
4+
5+
exports.up = pgm =>{
6+
pgm.sql(`CREATE TABLE posts (
7+
id SERIAL PRIMARY KEY,
8+
url VARCHAR(300),
9+
lat NUMERIC,
10+
lng NUMERIC
11+
);
12+
`);
13+
};
14+
15+
exports.down = pgm =>{
16+
pgm.sql(`
17+
DROP TABLE posts;
18+
`);
19+
};

0 commit comments

Comments
 (0)