aboutsummaryrefslogtreecommitdiffstats
path: root/src/extract_lambda.py
Commit message (Expand)AuthorAgeFilesLines
* feat: add more logging for debuggingAlex Schofield2024-08-201-2/+5
* refactor: change rows output to debug logger outputAlex Schofield2024-08-201-2/+1
* feat: add additional logging and exclude unnecessary tableAlex Schofield2024-08-201-5/+7
* feat: add advanced loggingAlex Schofield2024-08-201-5/+11
* feat: conditional logic for if bucket is emptyAlex Schofield2024-08-201-9/+23
* feat: revert s3 streaming to previous implementation for uploadingAlex Schofield2024-08-201-31/+25
* fix(extract_lambda): fix UnboundLocalError when db is called before it is ass...Alex Schofield2024-08-201-0/+1
* refactor: optimise s3 streaming & file namingAlex Schofield2024-08-191-11/+6
* feat: use buffers for s3 upload instead of csv filesAlex Schofield2024-08-191-17/+33
* refactor: remove print statements in process_and_upload_tablesAlex Schofield2024-08-191-2/+0
* refactor: add timestamp function in extract_lambda.pyAlex Schofield2024-08-191-12/+17
* chore: reorganise imports in extract_lambdaAlex Schofield2024-08-191-5/+5
* refactor(retrieve_secrets): add logging for ClientErrorAlex Schofield2024-08-191-0/+1
* refactor(retrieve_secrets): improve error handling when retrieving secretsAlex Schofield2024-08-191-0/+3
* refactor(lambda_handler): remove unnecessary else statementAlex Schofield2024-08-191-8/+7
* refactor(retrieve_secrets): use aws recommended method for retrieving secretsAlex Schofield2024-08-191-8/+11
* refactor(log): implement logging ancestry - avoid using root loggerAlex Schofield2024-08-191-1/+1
* Merge branch 'development' into feature/test-process-upload-tables-v2Alex2024-08-191-0/+1
|\
| * wip: fixed broken tests; hashed out test_error_retrieving_objectlian-manonog2024-08-191-0/+1
* | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-191-5/+3
* | add working process and upload tables testEllie2024-08-191-2/+2
* | wip: fixing last testEllie2024-08-191-2/+3
|/
* style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-161-1/+3
* dummy comment to test checksAng Bel2024-08-161-1/+1
* style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-161-26/+40
* refactoring to be more in line with pythonic code practices and prevent sql i...Ang Bel2024-08-161-6/+11
* refactor following github actions major risk messageAng Bel2024-08-161-34/+31
* style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-161-59/+71
* refactoring for extract lambda to filter by last updated and if not empty wri...Ang Bel2024-08-161-15/+11
* extract bucket name retrieval helper function and replace the bucket name pla...Ang Bel2024-08-161-4/+7
* Secrets manager integration into the extract lambda reviewed.Ang Bel2024-08-161-13/+16
* procefss_and_upload_tables test in progressAng Bel2024-08-151-9/+21
* Merge branch 'feature-extract-lambda-data-extraction' into tf-secrets-managerlian-manonog2024-08-151-21/+132
|\
| * test: mid-through test for process_and_upload_tablesT-Aji2024-08-141-9/+7
| * test: passing test for function connect_to_databaseT-Aji2024-08-141-15/+25
| * feat/passing tests to helper function list_existing_s3_filesT-Aji2024-08-141-5/+7
| * feat/ add logging & split task into 3 helper functionsT-Aji2024-08-131-31/+109
| * dumps data to csvT-Aji2024-08-131-5/+27
* | wip: secrets manager pushing to merge with extract_lambdalian-manonog2024-08-151-0/+1
|/
* database connection added to funcT-Aji2024-08-131-0/+32
* s3 buckets in tf and initial blank lambda py files set-up in src folder for t...Ang Bel2024-08-131-0/+0
git.ajschof.me — hosted by ajschofield — powered by cgit