✨ Add Async SQLAlchemy

[Kludex]

@long2ice Would you accept a PR for the SQLAlchemy 1.4 async?

This PR is not fully implemented (I got tired). But I'll continue later if you consider accepting it.

[long2ice]

Thanks! Every PR is great

[Kludex]

I'll continue tomorrow then. Thanks. :)

[Kludex]

@long2ice I've tested only with SQLite. The results were far too positive, so I'm not sure if I've made some mistakes around.

Maybe adding a virtual machine with limited resources will be more fair. What do you think?

Do you mind giving a quick look? Thank you in advance! 😗

[long2ice]

I have no more questions, thanks your contribution, if your finish this I will merge that.

[teemuy]

Is this PR still missing something as it hasn't been merged yet?

[long2ice]

Oh I miss that, is this PR ready? @Kludex

[Kludex]

Not that I'm aware. @long2ice is there something missing here?

[long2ice]

No, I mean if you finished this PR, I can merge it right now

[Kludex]

Let me check once again today. As far as I remember it was completed.

I'll confirm in some hours.

[Achilles0509]

I made test on my PC, async SqlAlchemyCore is slower than sync SqlAlchemyCore, and of course slower than Tortoise ORM.

Here is specification.
Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz
RAM 16G

• sqlite3

=============== ==================== ========== ========== ============== ========== ==========
Test 1 Async SQLAlchemy ORM peewee Pony ORM SQLAlchemy ORM SQLObject Max
=============== ==================== ========== ========== ============== ========== ==========
Insert: Single 403.14 6010.24 5244.13 1749.92 3499.13 6010.24
Insert: Batch 357.94 7346.70 22874.69 9532.73 5129.89 22874.69
Insert: Bulk 257.01 31279.77 — 26061.29 — 31279.77
Filter: Large 14751.03 40551.58 118237.67 81507.16 33187.12 118237.67
Filter: Small 10079.86 23120.13 12549.29 24950.05 30304.13 30304.13
Get 1841.26 3740.36 9048.22 3113.56 6914.62 9048.22
Filter: dict 20203.19 53316.07 94728.73 74851.50 — 94728.73
Filter: tuple 22722.07 52665.13 119659.48 104028.84 — 119659.48
Update: Whole 1370.52 6668.63 25919.56 18855.04 14431.52 25919.56
Update: Partial 1213.20 8889.50 37236.36 31846.60 30624.30 37236.36
Delete 22845.22 12638.01 59128.84 57288.80 4507.92 59128.84
Geometric Mean 2999.85 15225.01 31752.58 22470.17 11206.18 34783.85
=============== ==================== ========== ========== ============== ========== ==========

=============== ==================== ========== ========== ============== ========== ==========
Test 2 Async SQLAlchemy ORM peewee Pony ORM SQLAlchemy ORM SQLObject Max
=============== ==================== ========== ========== ============== ========== ==========
Insert: Single 391.72 5740.27 4428.81 1372.82 3294.04 5740.27
Insert: Batch 342.35 7167.67 15442.38 5529.88 4852.16 15442.38
Insert: Bulk 250.60 30838.20 — 11144.10 — 30838.20
Filter: Large 14278.47 35893.75 115583.77 66827.30 32693.29 115583.77
Filter: Small 10420.08 20548.79 15902.73 20748.43 29208.25 29208.25
Get 1784.84 3522.60 8696.01 2760.54 6720.94 8696.01
Filter: dict 19080.08 50213.35 84941.91 64198.53 — 84941.91
Filter: tuple 18896.50 48613.27 114122.96 85621.34 — 114122.96
Update: Whole 1044.44 5856.25 24710.17 12424.25 15061.69 24710.17
Update: Partial 1180.20 8412.50 33190.66 19150.32 30929.16 33190.66
Delete 19532.51 13047.53 22200.31 1046.18 2860.88 22200.31
Geometric Mean 2782.7 14315.03 26825.24 11449.06 10398.6 29396.2
=============== ==================== ========== ========== ============== ========== ==========

=============== ==================== ========== ========== ============== ========== ==========
Test 3 Async SQLAlchemy ORM peewee Pony ORM SQLAlchemy ORM SQLObject Max
=============== ==================== ========== ========== ============== ========== ==========
Insert: Single 325.10 2834.76 2739.14 1418.62 2120.69 2834.76
Insert: Batch 251.58 3192.28 5488.92 5494.03 2649.26 5494.03
Insert: Bulk 251.43 9621.73 — 14564.06 — 14564.06
Filter: Large 4361.75 13886.74 45638.88 29987.66 17548.25 45638.88
Filter: Small 3858.66 8991.47 6144.69 14826.86 16343.14 16343.14
Get 1062.82 1338.17 5547.80 2546.53 4215.57 5547.80
Filter: dict 4700.99 19095.34 23845.57 26055.89 — 26055.89
Filter: tuple 5006.40 20433.44 46184.38 36522.28 — 46184.38
Update: Whole 1329.32 1732.52 19406.84 14510.99 14521.70 19406.84
Update: Partial 1671.82 8442.41 24748.81 21856.72 27962.96 27962.96
Delete 24699.43 12881.24 51912.92 40192.01 2980.76 51912.92
Geometric Mean 1751.27 6619.65 15162.63 12836.37 7317.61 16885.65
=============== ==================== ========== ========== ============== ========== ==========

• MySql

=============== ========== ========== ============== ============ ==========
Test 1 peewee Pony ORM SQLAlchemy ORM Tortoise ORM Max
=============== ========== ========== ============== ============ ==========
Insert: Single 7.61 9.24 8.08 37.04 37.04
Insert: Batch 528.15 629.38 646.70 321.58 646.70
Insert: Bulk 481.03 — 581.86 243.87 581.86
Filter: Large 29364.54 128924.60 49279.63 40766.25 128924.60
Filter: Small 12439.52 10745.04 14658.77 22534.15 22534.15
Get 1738.70 5059.66 1921.57 3072.19 5059.66
Filter: dict 37842.66 99917.91 48611.01 48515.24 99917.91
Filter: tuple 37656.03 127155.99 61197.36 45381.44 127155.99
Update: Whole 582.84 943.77 1148.82 516.76 1148.82
Update: Partial 1063.93 1493.32 1793.63 801.52 1793.63
Delete 1810.05 1645.21 1672.55 1017.55 1810.05
Geometric Mean 1992.4 4054.59 2639.19 2253.4 4317.13
=============== ========== ========== ============== ============ ==========

=============== ========== ========== ============== ============ ==========
Test 2 peewee Pony ORM SQLAlchemy ORM Tortoise ORM Max
=============== ========== ========== ============== ============ ==========
Insert: Single 8.28 8.85 9.03 37.16 37.16
Insert: Batch 764.36 684.06 603.81 427.03 764.36
Insert: Bulk 546.78 — 650.69 298.70 650.69
Filter: Large 31356.16 125794.53 49121.30 27909.49 125794.53
Filter: Small 11845.49 12678.78 10039.04 20675.76 20675.76
Get 1645.99 5301.83 1900.27 2854.26 5301.83
Filter: dict 36464.92 91024.20 46892.55 44600.64 91024.20
Filter: tuple 37521.73 122633.30 54790.98 40059.96 122633.30
Update: Whole 740.19 1187.88 904.45 734.66 1187.88
Update: Partial 1578.47 1693.68 1425.47 1052.31 1693.68
Delete 1605.55 470.69 516.16 1121.16 1605.55
Geometric Mean 2185.38 3743.96 2194.49 2350.33 4295.62
=============== ========== ========== ============== ============ ==========

=============== ========== ========== ============== ============ ==========
Test 3 peewee Pony ORM SQLAlchemy ORM Tortoise ORM Max
=============== ========== ========== ============== ============ ==========
Insert: Single 7.85 8.23 8.63 33.79 33.79
Insert: Batch 236.46 187.78 199.81 168.07 236.46
Insert: Bulk 230.89 — 426.77 151.07 426.77
Filter: Large 10894.16 48883.81 25916.95 13625.92 48883.81
Filter: Small 5324.91 5285.44 11218.18 7934.37 11218.18
Get 654.85 3488.54 1578.16 1197.50 3488.54
Filter: dict 12867.33 24738.81 23004.13 15003.48 24738.81
Filter: tuple 13690.22 50263.39 30714.15 14109.17 50263.39
Update: Whole 550.02 1069.65 710.93 711.34 1069.65
Update: Partial 1764.65 1068.38 2140.78 943.97 2140.78
Delete 1113.26 1514.09 1909.48 1848.48 1909.48
Geometric Mean 1108.3 2226.74 1812.45 1358.93 2591.92
=============== ========== ========== ============== ============ ==========

• PostgreSql

=============== ========== ========== ============== ============ ==========
Test 1 peewee Pony ORM SQLAlchemy ORM Tortoise ORM Max
=============== ========== ========== ============== ============ ==========
Insert: Single 116.18 110.06 67.72 545.72 545.72
Insert: Batch 1403.53 2077.01 2216.03 1990.21 2216.03
Insert: Bulk 3808.16 — 2437.39 5775.37 5775.37
Filter: Large 49961.73 93299.02 49832.92 61402.82 93299.02
Filter: Small 12631.99 8950.33 13207.72 24769.51 24769.51
Get 1794.63 5325.29 1890.15 — 5325.29
Filter: dict 58564.12 77649.29 47666.52 — 77649.29
Filter: tuple 59667.74 93841.75 37475.58 — 93841.75
Update: Whole 2302.02 4387.46 3309.33 — 4387.46
Update: Partial 3349.83 4034.91 4521.72 — 4521.72
Delete 4606.11 4619.81 6130.74 — 6130.74
Geometric Mean 5271.18 7554.98 5196.23 6250.44 9755.67
=============== ========== ========== ============== ============ ==========

=============== ========== ============== ==========
Test 2 peewee SQLAlchemy ORM Max
=============== ========== ============== ==========
Insert: Single 117.08 — 117.08
Insert: Batch 1403.06 — 1403.06
Insert: Bulk 1495.83 2176.95 2176.95
Geometric Mean 626.35 2176.95 709.8
=============== ========== ============== ==========

[Kludex]

So this branch works just fine, right? 😗

[Achilles0509]

For PostgreSql there are many failed benchmark tests.
Can you pull the latest code to your repository and working branches?
Then I can make test again.

[Kludex]

Ah, I didn't notice you said that Async SQLAlchemy Core is slower. That doesn't make sense. 😗

I'm going to see if I can get this at some point this week, I'm not having much time.

[zhangkaizhao]

Hi! Maybe you can ask for some help from Mike Bayer - the author of SQLAlchemy. He has just made a pull request for the new ORM benchmark project called IMDBbench by EdgeDB.

[Kludex]

I said that it doesn't make sense because I remember my previous results giving a better performance as the one I'm providing here.

In any case, if @zzzeek wants to help, he is more than welcome...

I still didn't have the time to check this PR.

[zzzeek]

i would suggest doing some SQL echoing to see if the number of SQL calls is different. if you are comparing psycopg2 to asyncpg, SQLAlchemy has optimizations to INSERTs in place for psycopg2 only at the moment.