Create a Prod environment #1

jakubgs · 2019-07-25T01:47:28Z

We will need a proper prod environment for release of https://dap.ps/.

Things we will need:

Vital

[prod] Get an SSL Certificate for AWS ELB #2 - SSL Certificate that we can use with Elastic Load Balancer
[prod] Create Elastic Load Balancer setup #3 - Elastic Load Balancer for balancing load
[prod] Scalability with load #5 - Scalability with load
[prod] MongoDB cluster #6 - MongoDB cluster
[prod] Backups for MongoDB #7 - Backups for MongoDB

Optional

[prod] Geographically spread hosts #4 - Geographically spread hosts
[prod] Periodic EC2 Snapshots #10 - Periodic EC2 snapshots
[prod] MongoDB Web UI #11 - WebUI for MongoDB

This is enough for for a week if I have nothing else to do.
Especially with the scalability and geographical spread.

jakubgs · 2019-07-25T01:48:50Z

@andytudhope questions:

What is the deadline for prod?
Which of those are essential and which are optional?
How much money can I spend on this infra?

andytudhope · 2019-07-25T14:21:58Z

Deadline: next Tuesday, 30/07/2019.
All of them are essential, except for the web UI.
Lmk what 3 hosts and all the DB stuff costs, I would say no more than $300/month if possible, but not really sure about costs on AWS these days.

jakubgs · 2019-07-25T14:43:34Z

Tuesday, 30/07/2019.

That's gonna be tough. We'll see.

jakubgs · 2019-07-25T15:57:55Z

This repo could be helpful in figuring out how to do this in Terraform:
https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment

jakubgs · 2019-07-25T17:46:45Z

I consider auto-scaling to be the hardest part of this and will consider it non-essential for now.
It can be added later.

Scratch that, I'm getting auto-scaling for free by suing CloudPosse module, but the geographical spread will be harder. For now I'm deploying in eastern USA.

jakubgs · 2019-07-30T17:17:51Z

As https://dap.ps/ is currently hosted on GitHub I will be not touching that unless we are ready.

I will deploy the prod environment under https://prod.dap.ps/, later we can do a CNAME from dap.ps.

jakubgs · 2019-07-30T17:38:24Z

Fuck...

I used terraform state mv to rename a resource in infra-dapps but I forgot that I should be using terraform_0.11 and it used 0.12... and upgraded my state to 0.12.

 $ terraform_0.11 refresh

Error: 
Terraform doesn't allow running any operations against a state
that was written by a future Terraform version.

And now I'm fucked because cloudposse/terraform-aws-elastic-beanstalk-environment doesn't support Terraform 0.12.

So either I remove everything from AWS and re-create it, which is stupid, or I have to write a 0.12 compatible version of the CloudPosse module. Fuck my life.

jakubgs · 2019-07-30T17:44:46Z

Opened an issue with Terraform about no confirmation prompt, but it is my fault non-the-less:
hashicorp/terraform#22261

andytudhope · 2019-07-30T17:49:05Z

ok, sorry to hear it isn't going well and still appreciate all the effort this is taking.

jakubgs · 2019-07-30T17:54:35Z

Okay, there's hope, someone already made a basic 0.12 compatible version of the the module:
https://github.com/lodotek/terraform-aws-elastic-beanstalk-environment
Blessed be @lodotek from cloudposse/terraform-aws-elastic-beanstalk-environment#86 (comment)

jakubgs · 2019-07-30T18:37:41Z

Ok, managed to fix it by using two forks from @lodotek and some other newer versions: dcf66cd

ok, sorry to hear it isn't going well and still appreciate all the effort this is taking.

Raging at technology is my daily bread, nothing out of the ordinary.
Here's is my preferred mode of transportation:

jakubgs · 2019-07-30T19:09:30Z

Okay, I've created a auto-scalable prod environment available under https://prod.dap.ps/ in dc4151b.

Next step is the MongoDB cluster, once that's up prod infra should be ready sans geo spread.

@andytudhope are there different values for Prod for these variables?

BLOCKCHAIN_CONNECTION_POINT
DISCOVER_CONTRACT

I assume so?

jakubgs · 2019-08-01T13:00:58Z

Looks like I found the issue with this error:

./src/common/blockchain/services/discover-service/discover-service.js
Module not found: Can't resolve '../../../../embarkArtifacts/contracts/Discover' in
'/var/app/current/src/common/blockchain/services/discover-service'

I SSHed into the EB host and checked if the file exists:

[ec2-user@ip-10-0-102-79 ~]$ cd /var/app/current/src/common/blockchain/services/discover-service

[ec2-user@ip-10-0-102-79 discover-service]$ ls -l
total 16
-rw-r--r-- 1 nodejs nodejs 8242 Aug  1 05:48 discover-service.js
-rw-r--r-- 1 nodejs nodejs 1909 Aug  1 05:48 discover-validator.js

[ec2-user@ip-10-0-102-79 discover-service]$ ls -l ../../../../embarkArtifacts/contracts/Discover.js
ls: cannot access ../../../../embarkArtifacts/contracts/Discover.js: No such file or directory

And it did not, so then I remembered that:

EB CLI ignores files specified in .ebignore, and includes all other files. In particular, it includes uncommitted source files.

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb3-deploy.html

And there's a --staged flag that will:

Deploy files staged in the git index instead of the HEAD commit.

The correct build process for dev.dap.ps is simply:

yarn instal
./node_modules/.bin/embark build testnet
eb init
git add --force  src/embarkArtifacts
eb deploy --staged

But that only got me a new error:

./src/embarkArtifacts/embarkjs.js
Module not found: You attempted to import /home/sochan/work/dapps:discover/src/embarkArtifacts/config/blockchain.json which falls outside of the project src/ directory. Relative imports outside of src/ are not supported.

Because the src/embarkArtifacts/embarkjs.js file uses absolute paths:

 $ head -n 3 src/embarkArtifacts/embarkjs.js   
/* eslint-disable */

const EmbarkJS = require("/home/sochan/work/dapps:discover/src/embarkArtifacts/modules/embarkjs").default;

jakubgs · 2019-08-01T13:22:18Z

I have verified that the ElasticBeanstalk environment does scale with CPU load.

jakubgs · 2019-08-01T18:30:26Z

I have finished deploying the prod MongoDB replication set(cluster) and verified it's working.

I've updated the DB_CONNECTION environment variable in the prod-dap-ps-app so it's ready.

jakubgs · 2019-08-01T19:16:11Z

I fixed the absolute paths generated by Embark by hand(well, mostly with sed) and deployed that to Dev environment and it is up right now: https://dev.dap.ps/

But when I tried submitting my Dapp I got back:

A DApp was not uploaded in the client

And here's the reason:

Looks like the /metadata request is made to http://dev.dap.ps/metadata, not using HTTPS.
But when I used the http://dev.dap.ps/ endpoint without SSL I got back 404:

jakubgs · 2019-08-05T20:26:01Z

I've prepared some fixes for the app itself: dap-ps/discover#25

And once I have some funds for the 0x9fb115BC152aE21ECDc81c73843673fa38e4D2dB contract I can try creating a submission.

jakubgs · 2019-08-06T15:52:48Z

Email sending issues fixed in #9.

jakubgs · 2019-08-06T15:57:36Z

Looks like it works:

jakubgs · 2019-08-06T19:22:05Z

Yay, my TEH_REI crypto Dapp is up!

Glorious.

jakubgs · 2019-08-07T15:31:42Z

Updated the prod configuration with Mainnet contract deployed by Andy: d344fef

jakubgs · 2019-08-07T16:26:31Z

Okay, Apparently there is and environment variable named EB_NODE_COMMAND which is by default for Node.js just takes the value of the start task from package.json. I think if I set it explicitly for the environment I will be able to change start target to the old value of ./node_modules/.bin/react-scripts start.

jakubgs · 2019-08-07T16:36:54Z

Brilliant, create-react-app is literally incapable for building a dev build with react-scripts build:
facebook/create-react-app#2880
Which means I can't use React Developer Tools with https://dev.dap.ps/ because I can't build a dev build for it.
The retardation never stops.

jakubgs · 2019-08-07T16:40:42Z

Also we are getting 403 from a request to ipfs.infura.io:

https://ipfs.infura.io:5001/api/v0/id?stream-channels=true
ipfs method not allowed

No clue what that is about.

jakubgs · 2019-08-07T16:43:42Z

Mixed content issue: #12

jakubgs · 2019-08-07T17:35:53Z

@andytudhope looks like you broke this application in
dap-ps/discover@433afce#diff-5e5eb47cd24f028b901ce69c09dcaa14R1

You replaced:

const wallet = require('./development/mnemonic')

With:

const wallet = require('./livenet/mnemonic')

But there is no such thing as config/livenet:
https://github.com/dap-ps/discover/tree/master/config/livenet

How is this supposed to work?

jakubgs · 2019-08-07T17:40:16Z

@andytudhope also, why are you merging PRs like dap-ps/discover#28 without me reviewing them first? What's the point of making a PR if nobody is going to review it? You might as well commit directly to master if you don't wait for reviews.

andytudhope · 2019-08-07T18:22:41Z

sorry, you're right. Will change that back.

andytudhope · 2019-08-07T19:53:53Z

Also we are getting 403 from a request to ipfs.infura.io:

This is because there is a bug in Embark which means that you have to change the embarkjs.js file generated in src/embarkArtifacts after running embark build (it's detailed in the README). I know it's a nasty AF hack, but there really is no way around it, and the Embark team have yet to release 4.1 with the fix...

jakubgs · 2019-08-07T22:52:20Z

Replacement of build.sh with a Makefiles: dap-ps/discover#3

jakubgs · 2019-08-08T13:47:07Z

Mixed content issue #12 fixed in dap-ps/discover#7.

jakubgs · 2019-08-08T17:53:44Z

Yet another issue that needed fixing: dap-ps/discover#9
Not handling paths that are not in frontend folder on the backend.
And the fix: dap-ps/discover#10

jakubgs · 2019-08-09T13:10:57Z

There is one more thing I need to do.. Currently the certificate issues by AMC only handles the specific domain in question:

infra-dapps/modules/aws-eb-env/cert.tf

Lines 1 to 7 in d344fef

    
           resource "aws_acm_certificate" "main" { 
        
             domain_name = "${var.stage}.${var.dns_domain}" 
        
             /* TODO support SAN of dap.ps */ 
        
             subject_alternative_names = [] 
        
             validation_method         = "DNS" 
        
           }

So I need to modify this to include dap.ps. Either as a SAN or make it the main domain and add the *.dap.ps subdomains as SANs.

I also have two options:

Both certificates for dev and prod should include dap.ps as a SAN
Only prod certificate will include dap.ps in valid domains

The first option would allow us to point dap.ps at dev.dap.ps, not sure if would want to do that.

jakubgs · 2019-08-09T16:31:15Z

Okay, I've re-created the Prod env with the certificate that includes the dap.ps domain in the SANs:

Here's the change: d6853ee
But I will have to do the same thing to Dev, which will make it unavailable for a while.

jakubgs · 2019-08-09T20:09:15Z

Both certs have been re-issued with dap.ps in their SANs:

jakubgs · 2019-08-14T13:17:17Z

As we have prod up and running, can I close this @andytudhope ? I have the other 3 optional issues open.

andytudhope · 2019-08-14T14:56:45Z

Yeah, just that there is an error on prod.dap.ps/metadata/all still and I'm not sure why...

jakubgs · 2019-08-14T19:43:55Z

That was due to lack of ?replicaSet= argument in the URI in DB_CONNECTION env in prod, fixed.

jakubgs self-assigned this Jul 25, 2019

This was referenced Jul 25, 2019

[prod] Get an SSL Certificate for AWS ELB #2

Closed

[prod] Geographically spread hosts #4

Open

[prod] MongoDB cluster #6

Closed

[prod] Scalability with load #5

Closed

[prod] Create Elastic Load Balancer setup #3

Closed

[prod] Backups for MongoDB #7

Closed

jakubgs mentioned this issue Aug 7, 2019

[prod] Fix "Mixed Content" browser warnings #12

Closed

jakubgs closed this as completed Aug 14, 2019

jakubgs mentioned this issue Jul 22, 2021

Renew iOS Fastlane match Profiles status-im/status-mobile#8745

Closed

Create a Prod environment #1

Create a Prod environment #1

Comments

jakubgs commented Jul 25, 2019 • edited Loading

Vital

Optional

jakubgs commented Jul 25, 2019

andytudhope commented Jul 25, 2019

jakubgs commented Jul 25, 2019

jakubgs commented Jul 25, 2019

jakubgs commented Jul 25, 2019 • edited Loading

jakubgs commented Jul 30, 2019

jakubgs commented Jul 30, 2019

jakubgs commented Jul 30, 2019

andytudhope commented Jul 30, 2019

jakubgs commented Jul 30, 2019

jakubgs commented Jul 30, 2019

jakubgs commented Jul 30, 2019

jakubgs commented Aug 1, 2019

jakubgs commented Aug 1, 2019

jakubgs commented Aug 1, 2019

jakubgs commented Aug 1, 2019 • edited Loading

jakubgs commented Aug 5, 2019

jakubgs commented Aug 6, 2019

jakubgs commented Aug 6, 2019

jakubgs commented Aug 6, 2019

jakubgs commented Aug 7, 2019

jakubgs commented Aug 7, 2019

jakubgs commented Aug 7, 2019 • edited Loading

jakubgs commented Aug 7, 2019

jakubgs commented Aug 7, 2019

jakubgs commented Aug 7, 2019

jakubgs commented Aug 7, 2019

andytudhope commented Aug 7, 2019

andytudhope commented Aug 7, 2019

jakubgs commented Aug 7, 2019

jakubgs commented Aug 8, 2019

jakubgs commented Aug 8, 2019

jakubgs commented Aug 9, 2019

jakubgs commented Aug 9, 2019

jakubgs commented Aug 9, 2019

jakubgs commented Aug 14, 2019

andytudhope commented Aug 14, 2019

jakubgs commented Aug 14, 2019

jakubgs commented Jul 25, 2019 •

edited

Loading

jakubgs commented Jul 25, 2019 •

edited

Loading

jakubgs commented Aug 1, 2019 •

edited

Loading

jakubgs commented Aug 7, 2019 •

edited

Loading