forked from apache/libcloud
-
Notifications
You must be signed in to change notification settings - Fork 0
Azure ARM deployment #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
famanson
wants to merge
131
commits into
trunk
Choose a base branch
from
azure-arm
base: trunk
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
131 commits
Select commit
Hold shift + click to select a range
d4443a6
Start work on Azure ARM
b2782a7
Update AzureResourceManagerConnection
7131c77
Refactor machine types and initialiser for the new driver
c79f44d
Update the host
cec35c5
Copy pasta the HTTP request code
6280dca
Add some placeholders
a86a796
Add _default_path_prefix
49cbeb9
Add API version by default
fde6eb9
Completed method list_locations
Denvar94 2c7cf8f
Fix cyclic dependencies
23123fa
Remove the redundant class
2461c0a
Fix this path
2639e35
Added mapping of json to Node Location method
Denvar94 d80706c
Merge conflict resovled
Denvar94 3433fc7
Using property decorator
Denvar94 f6d8072
Clean up
Denvar94 e1fc469
Uses NodeLocation for the new API
Denvar94 4a7c2d9
Use a proper JSON response
d80af53
Fix typo
7800e05
Use single quotes
8868ad2
Add list_sizes
1fc425c
For now, allow using different API versions
ce7c893
Fix wrong key
4f5d8ca
Added list_nodes function
Denvar94 82c575b
Added documentation
Denvar94 6a1bfeb
Add create_node method
6e44b9b
Create a public IP address
8c9af5e
Create a public IP address
3f44a85
Finish create_node call
42cde54
Parse body before returning
d194f5f
clean up"
Denvar94 91a7e79
Merge branch 'azure-arm' of github.com:esplorio/libcloud into azure-arm
Denvar94 e62e466
Use location.id
97c21ac
Fix typo
9e704c3
API Version update
436548d
Try an image ref instead
b08b19a
Update ssh key path
8f21918
Populate the Node to retun list of nodes
Denvar94 96db814
Merge branch 'azure-arm' of github.com:esplorio/libcloud into azure-arm
Denvar94 9617866
Fix data queries
b03dd1e
Return a node on create_node
11133bb
Return an empty node
04ac546
Reconstruct the URL
7c0f5c8
This is actually a list
d0605e6
Put IP addresses in the right place
07c6894
Added call to retreive public ip address
Denvar94 830d4c2
Use static allocation
e06ba37
Add missing property
aa31c67
Make these dynamic now
fa9b7da
Fix typo
2493868
Add distinction between these methods
26bfdfa
Add NodeState config
8bf08f2
Add error output
a480dd7
Add availability set
69dd20a
Remove typo
d41413d
Added comments to get methods
Denvar94 2a026a0
Add a domainNameLabel setting
2f1c08c
Add data disk option
43c7102
Merge branch 'azure-arm' of github.com:esplorio/libcloud into azure-arm
8d39228
Use ReadWrite cache option
7b7c375
Add missing key
c12d454
Add retry handling for rate limit errors
c95734f
Add explicit None check for retries
de13e7e
Forgot a return
890008b
Add comment
427b223
Clean up lint errors
Denvar94 75bfb77
Limited character space in line to 79
Denvar94 d6c74ef
Update comment
ea680f4
Update how to create a token
359ccfb
Merge branch 'azure-arm' of github.com:esplorio/libcloud into azure-arm
22876a5
Add skeleton code for Azure Arm tests
Denvar94 fe1205b
Merge branch 'azure-arm' of github.com:esplorio/libcloud into azure-arm
Denvar94 ef09f39
List nodes without the need of a resource groups
Denvar94 062d1b2
Added set up method
Denvar94 b99ed02
Added test method for listing locations
Denvar94 6861b4e
Added method paths for testing
Denvar94 20e69ca
Added test data for tests
Denvar94 7ae09b5
Fixed list location method and added test
Denvar94 4c83263
Fixed urls in the sample jsons
Denvar94 c6e8819
Added methods to mock the requests for network information and amende…
Denvar94 58eef3c
Added test to check node addresses
Denvar94 5b9ff7f
Added class and methods to list azure images
Denvar94 ad5175d
Clean up
Denvar94 864a6c7
Added os propety to Azure image
Denvar94 b3101a7
Add azure image and public ip option
Denvar94 68b95e2
Added azure network, subnet and config classes
Denvar94 c73538f
Clean up
Denvar94 f0dbab1
Fix missing :
Denvar94 26ea3f1
Use network configuation in create node and create nic methods
Denvar94 e6f8a02
Clean up
Denvar94 9c1638e
Added reboot node method and perform post
Denvar94 c419a32
Fix typo
57e57c3
Added method to return the state of VM
Denvar94 7885f3e
Changed parameters of get state of node
Denvar94 35a5976
Added return value for reboot_node
Denvar94 b967446
Added destroy node method
Denvar94 6c70449
Fix typos
c8b347a
Fix URL
771b8ab
Add value access
0d8a041
Remove another typo
eebe390
Fix more typos
d2d678e
Missing list index
99fb070
Use a proper payload
ea75c79
Remove location/location.id ambiguity
d1bebbd
Remove confusing comment
ce5e619
Add missing path
388af05
Fix publisher/offer access
964b201
Update comparison
b7917f1
Do not return prematurely
a03cef7
Made changes to pass tox tests
Denvar94 d04acc1
Reordered functions to fit libcloud criteria
Denvar94 1eb4262
Renamed methods to fit libcloud criteria
Denvar94 7e91283
Add tests
Denvar94 1e0afaa
Amended parameter name in list_nodes to fit with libcloud format
Denvar94 6f93b33
Bug fixes and typos
Denvar94 5935c56
Updated method name to be internal
Denvar94 c7a9076
Added python 3 tests
Denvar94 43c53f3
Change method name
Denvar94 f763747
Added test fixtures for azure arm requests
Denvar94 8836cac
Added tests for list images
Denvar94 8f32c66
Added test for list sizes
Denvar94 59db36f
Added create vm test
Denvar94 10c89bf
Fixed create vm test
Denvar94 15ef6c9
Fixed node id issues
Denvar94 9054168
Get rid of redundant tests
Denvar94 20d7647
Use the correct node ID
c56dc90
Merge branch 'azure-arm' of github.com:esplorio/libcloud into azure-arm
1c9265a
Rewrite the script to allow for multiple storage accounts
5a9afcf
Remove ex_storage_account_name
29173e0
Fix the config
086a670
Assign a different LUN value
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,6 +20,7 @@ | |
| import hmac | ||
|
|
||
| from hashlib import sha256 | ||
|
|
||
| from libcloud.utils.py3 import httplib | ||
| from libcloud.utils.py3 import b | ||
| from libcloud.utils.xml import fixxpath | ||
|
|
@@ -31,7 +32,8 @@ | |
|
|
||
| from libcloud.common.types import InvalidCredsError | ||
| from libcloud.common.types import LibcloudError, MalformedResponseError | ||
| from libcloud.common.base import ConnectionUserAndKey, RawResponse | ||
| from libcloud.common.base import ConnectionUserAndKey, RawResponse, \ | ||
| Connection, JsonResponse | ||
| from libcloud.common.base import CertificateConnection | ||
| from libcloud.common.base import XmlResponse | ||
|
|
||
|
|
@@ -42,6 +44,258 @@ | |
| AZURE_TIME_FORMAT = '%a, %d %b %Y %H:%M:%S GMT' | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does Microsoft offer a convention for the date format in their doc or is this just any date? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Again this is classic azure stuff. |
||
|
|
||
|
|
||
| """ | ||
| Sizes must be hardcoded because Microsoft doesn't provide an API to fetch them | ||
| From http://msdn.microsoft.com/en-us/library/windowsazure/dn197896.aspx | ||
|
|
||
| Prices are for Linux instances in East US data center. To see what pricing will | ||
| actually be, visit: | ||
| http://azure.microsoft.com/en-gb/pricing/details/virtual-machines/ | ||
| """ | ||
| AZURE_COMPUTE_INSTANCE_TYPES = { | ||
| 'A0': { | ||
| 'id': 'ExtraSmall', | ||
| 'name': 'Extra Small Instance', | ||
| 'ram': 768, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.0211', | ||
| 'max_data_disks': 1, | ||
| 'cores': 'Shared' | ||
| }, | ||
| 'A1': { | ||
| 'id': 'Small', | ||
| 'name': 'Small Instance', | ||
| 'ram': 1792, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.0633', | ||
| 'max_data_disks': 2, | ||
| 'cores': 1 | ||
| }, | ||
| 'A2': { | ||
| 'id': 'Medium', | ||
| 'name': 'Medium Instance', | ||
| 'ram': 3584, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.1266', | ||
| 'max_data_disks': 4, | ||
| 'cores': 2 | ||
| }, | ||
| 'A3': { | ||
| 'id': 'Large', | ||
| 'name': 'Large Instance', | ||
| 'ram': 7168, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.2531', | ||
| 'max_data_disks': 8, | ||
| 'cores': 4 | ||
| }, | ||
| 'A4': { | ||
| 'id': 'ExtraLarge', | ||
| 'name': 'Extra Large Instance', | ||
| 'ram': 14336, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.5062', | ||
| 'max_data_disks': 16, | ||
| 'cores': 8 | ||
| }, | ||
| 'A5': { | ||
| 'id': 'A5', | ||
| 'name': 'Memory Intensive Instance', | ||
| 'ram': 14336, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.2637', | ||
| 'max_data_disks': 4, | ||
| 'cores': 2 | ||
| }, | ||
| 'A6': { | ||
| 'id': 'A6', | ||
| 'name': 'A6 Instance', | ||
| 'ram': 28672, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.5273', | ||
| 'max_data_disks': 8, | ||
| 'cores': 4 | ||
| }, | ||
| 'A7': { | ||
| 'id': 'A7', | ||
| 'name': 'A7 Instance', | ||
| 'ram': 57344, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '1.0545', | ||
| 'max_data_disks': 16, | ||
| 'cores': 8 | ||
| }, | ||
| 'A8': { | ||
| 'id': 'A8', | ||
| 'name': 'A8 Instance', | ||
| 'ram': 57344, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '2.0774', | ||
| 'max_data_disks': 16, | ||
| 'cores': 8 | ||
| }, | ||
| 'A9': { | ||
| 'id': 'A9', | ||
| 'name': 'A9 Instance', | ||
| 'ram': 114688, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '4.7137', | ||
| 'max_data_disks': 16, | ||
| 'cores': 16 | ||
| }, | ||
| 'A10': { | ||
| 'id': 'A10', | ||
| 'name': 'A10 Instance', | ||
| 'ram': 57344, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '1.2233', | ||
| 'max_data_disks': 16, | ||
| 'cores': 8 | ||
| }, | ||
| 'A11': { | ||
| 'id': 'A11', | ||
| 'name': 'A11 Instance', | ||
| 'ram': 114688, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '2.1934', | ||
| 'max_data_disks': 16, | ||
| 'cores': 16 | ||
| }, | ||
| 'D1': { | ||
| 'id': 'Standard_D1', | ||
| 'name': 'D1 Faster Compute Instance', | ||
| 'ram': 3584, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.0992', | ||
| 'max_data_disks': 2, | ||
| 'cores': 1 | ||
| }, | ||
| 'D2': { | ||
| 'id': 'Standard_D2', | ||
| 'name': 'D2 Faster Compute Instance', | ||
| 'ram': 7168, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.1983', | ||
| 'max_data_disks': 4, | ||
| 'cores': 2 | ||
| }, | ||
| 'D3': { | ||
| 'id': 'Standard_D3', | ||
| 'name': 'D3 Faster Compute Instance', | ||
| 'ram': 14336, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.3965', | ||
| 'max_data_disks': 8, | ||
| 'cores': 4 | ||
| }, | ||
| 'D4': { | ||
| 'id': 'Standard_D4', | ||
| 'name': 'D4 Faster Compute Instance', | ||
| 'ram': 28672, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.793', | ||
| 'max_data_disks': 16, | ||
| 'cores': 8 | ||
| }, | ||
| 'D11': { | ||
| 'id': 'Standard_D11', | ||
| 'name': 'D11 Faster Compute Instance', | ||
| 'ram': 14336, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.251', | ||
| 'max_data_disks': 4, | ||
| 'cores': 2 | ||
| }, | ||
| 'D12': { | ||
| 'id': 'Standard_D12', | ||
| 'name': 'D12 Faster Compute Instance', | ||
| 'ram': 28672, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.502', | ||
| 'max_data_disks': 8, | ||
| 'cores': 4 | ||
| }, | ||
| 'D13': { | ||
| 'id': 'Standard_D13', | ||
| 'name': 'D13 Faster Compute Instance', | ||
| 'ram': 57344, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '0.9038', | ||
| 'max_data_disks': 16, | ||
| 'cores': 8 | ||
| }, | ||
| 'D14': { | ||
| 'id': 'Standard_D14', | ||
| 'name': 'D14 Faster Compute Instance', | ||
| 'ram': 114688, | ||
| 'disk': 127, | ||
| 'bandwidth': None, | ||
| 'price': '1.6261', | ||
| 'max_data_disks': 32, | ||
| 'cores': 16 | ||
| }, | ||
| 'Standard_D2_v2': { | ||
| 'id': 'Standard_D2_v2', | ||
| 'name': 'D2 v2 Faster Compute Instance', | ||
| 'ram': 7168, | ||
| 'disk': 100, | ||
| 'bandwidth': None, | ||
| 'price': '0.16', | ||
| 'max_data_disks': 4, | ||
| 'cores': 2 | ||
| }, | ||
| 'Standard_D3_v2': { | ||
| 'id': 'Standard_D3_v2', | ||
| 'name': 'D3 v2 Faster Compute Instance', | ||
| 'ram': 14336, | ||
| 'disk': 200, | ||
| 'bandwidth': None, | ||
| 'price': '0.319', | ||
| 'max_data_disks': 8, | ||
| 'cores': 4 | ||
| }, | ||
| 'Standard_D4_v2': { | ||
| 'id': 'Standard_D4_v2', | ||
| 'name': 'D4 v2 Faster Compute Instance', | ||
| 'ram': 28672, | ||
| 'disk': 400, | ||
| 'bandwidth': None, | ||
| 'price': '0.672', | ||
| 'max_data_disks': 16, | ||
| 'cores': 8 | ||
| }, | ||
| 'Standard_D5_v2': { | ||
| 'id': 'Standard_D5_v2', | ||
| 'name': 'D5 v2 Faster Compute Instance', | ||
| 'ram': 57344, | ||
| 'disk': 800, | ||
| 'bandwidth': None, | ||
| 'price': '1.277', | ||
| 'max_data_disks': 32, | ||
| 'cores': 16 | ||
| } | ||
| } | ||
|
|
||
|
|
||
| class AzureRedirectException(Exception): | ||
|
|
||
| def __init__(self, response): | ||
|
|
@@ -293,3 +547,53 @@ def add_default_headers(self, headers): | |
| headers['x-ms-date'] = time.strftime(AZURE_TIME_FORMAT, time.gmtime()) | ||
| # headers['host'] = self.host | ||
| return headers | ||
|
|
||
|
|
||
| class AzureARMResponse(JsonResponse): | ||
|
|
||
| valid_response_codes = [ | ||
| httplib.NOT_FOUND, | ||
| httplib.CONFLICT, | ||
| httplib.BAD_REQUEST, | ||
| httplib.TEMPORARY_REDIRECT | ||
| # added TEMPORARY_REDIRECT as this can sometimes be | ||
| # sent by azure instead of a success or fail response | ||
| ] | ||
|
|
||
| def success(self): | ||
| i = int(self.status) | ||
| return 200 <= i <= 299 or i in self.valid_response_codes | ||
|
|
||
|
|
||
| class AzureResourceManagerConnection(Connection): | ||
| driver = AzureBaseDriver | ||
| responseCls = AzureARMResponse | ||
| rawResponseCls = AzureRawResponse | ||
| name = 'Azure Resource Manager API Connection' | ||
| host = 'management.azure.com' | ||
| token = '' | ||
|
|
||
| def __init__(self, subscription_id, token, *args, **kwargs): | ||
| """ | ||
| :param subscription_id: Azure subscription ID. | ||
| :type subscription_id: ``str`` | ||
|
|
||
| :param token: JSON web token to authenticate with | ||
| Azure Active Directory | ||
| :type token: ``str`` | ||
| """ | ||
|
|
||
| super(AzureResourceManagerConnection, self).__init__( | ||
| *args, | ||
| **kwargs | ||
| ) | ||
| self.subscription_id = subscription_id | ||
| self.token = token | ||
|
|
||
| def add_default_headers(self, headers): | ||
| """ | ||
| @inherits: :class:`Connection.add_default_headers` | ||
| """ | ||
| headers['Content-Type'] = 'application/json' | ||
| headers['Authorization'] = 'Bearer %s' % self.token | ||
| return headers | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this
import b?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know this is classic azure stuff.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah sorry looked at wrong file