Skip to content

Commit

Permalink
Remove sleep call in pcfunc tile fetching (#132)
Browse files Browse the repository at this point in the history
The sleep call did have an effect of spreading out requests over time,
but in an arbitrary and non-efficient manner. The semaphore context
block effectively blocks on a counter-based lock.
  • Loading branch information
mmcfarland authored Oct 25, 2022
1 parent b4912be commit cd4b560
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 8 deletions.
6 changes: 3 additions & 3 deletions pc-funcs.dev.env
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ WEBSITE_HOSTNAME=funcs:8083

ANIMATION_OUTPUT_STORAGE_URL="http://azurite:10000/devstoreaccount1/output/animations"
ANIMATION_OUTPUT_ACCOUNT_KEY="Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
ANIMATION_API_ROOT_URL="https://planetarycomputer.microsoft.com/api/data/v1"
ANIMATION_API_ROOT_URL="https://planetarycomputer-staging.microsoft.com/api/data/v1"
ANIMATION_TILE_REQUEST_CONCURRENCY=2

IMAGE_OUTPUT_STORAGE_URL="http://azurite:10000/devstoreaccount1/output/images"
IMAGE_OUTPUT_ACCOUNT_KEY="Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
IMAGE_API_ROOT_URL="https://planetarycomputer.microsoft.com/api/data/v1"
IMAGE_TILE_REQUEST_CONCURRENCY=2
IMAGE_API_ROOT_URL="https://planetarycomputer-staging.microsoft.com/api/data/v1"
IMAGE_TILE_REQUEST_CONCURRENCY=2
5 changes: 0 additions & 5 deletions pcfuncs/funclib/tiles.py
Original file line number Diff line number Diff line change
Expand Up @@ -162,11 +162,6 @@ async def _f() -> io.BytesIO:
async with aiohttp.ClientSession() as session:
async with self._async_limit:
async with session.get(url) as resp:
# Download the image tile, block if exceeding concurrency limits
if self._async_limit.locked():
logger.info("Concurrency limit reached, waiting...")
await asyncio.sleep(1)

if resp.status == 200:
return io.BytesIO(await resp.read())
else:
Expand Down

0 comments on commit cd4b560

Please sign in to comment.