aboutsummaryrefslogtreecommitdiffstats
path: root/src
Commit message (Expand)AuthorAgeFilesLines
* Merge branch 'test/test_transform_lambda' into test/dataframesbulve-ad2024-08-236-248/+194
|\
| * wip: pushing againlian-manonog2024-08-232-6/+7
| * Merge pull request #94 from ajschofield/feature/transform-fact-sales-orderlian-manonog2024-08-232-5/+484
| |\
| | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-3/+6
| | * trying to resolce git index issue conflicts - commiting was the only solutionlian-manonog2024-08-231-9/+4
| | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-232-157/+236
| | * Deleted the fact_table schema py fileslian-manonog2024-08-234-233/+157
| * | fix: improve error handling for list_existing_s3_files and testsAlex Schofield2024-08-221-6/+10
| * | fix: handle no buckets properlyAlex Schofield2024-08-221-0/+3
| * | fix: change default parameters for bucket_name and clientAlex Schofield2024-08-221-2/+9
* | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-96/+154
* | | tests: additional tests written (pass) for dim tables transformation. Fact tr...Ang Bel2024-08-231-14/+16
* | | fix: refactoring for create_dim_date to include all date columns from all fac...Ang Bel2024-08-231-20/+13
* | | removed duplicate functionsT-Aji2024-08-231-89/+28
| |/ |/|
* | dataframes combined into one fileT-Aji2024-08-231-0/+238
* | Merge branch 'feature/transform-fact-sales-order' of https://github.com/ajsch...T-Aji2024-08-232-5/+53
|\ \
| * | wip: wrote pseudocode for lambda handler in writing df to parquet file format...lian-manonog2024-08-222-5/+53
* | | payment schema addedT-Aji2024-08-232-2/+46
|/ /
* | complete code for tables for purchase schema including a scrape for currency...Ang Bel2024-08-221-18/+48
* | add fact tableT-Aji2024-08-221-50/+0
* | Merge branch 'feature/transform-fact-sales-order' of https://github.com/ajsch...T-Aji2024-08-221-23/+30
|\ \
| * | refactored approach to writing transformation as functions per df. WIPAng Bel2024-08-221-23/+30
* | | add fact tableT-Aji2024-08-221-4/+31
|/ /
* | passing tests create_dim_design and create_dim_staffT-Aji2024-08-221-0/+50
* | refactor df creation into funcT-Aji2024-08-221-70/+34
* | wip: creating dim dataframesT-Aji2024-08-223-9/+58
|\ \
| * | wip: fact_purchase_order schemalian-manonog2024-08-213-3/+37
| * | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-211-18/+68
* | | began dim_date dfT-Aji2024-08-211-18/+17
|/ /
* | 5 dim tables createdT-Aji2024-08-211-15/+33
* | add dim tables: design, staff, currency, location (wip)Ellie2024-08-211-0/+54
|/
* Merge branch 'development' into feature/transform_lambdabulve-ad2024-08-214-96/+86
|\
| * chore: tidy-up repository & remove unused filesAlex Schofield2024-08-201-49/+0
| * feat: add more logging for debuggingAlex Schofield2024-08-201-2/+5
| * refactor: change rows output to debug logger outputAlex Schofield2024-08-201-2/+1
| * feat: add additional logging and exclude unnecessary tableAlex Schofield2024-08-201-5/+7
| * feat: add advanced loggingAlex Schofield2024-08-201-5/+11
| * feat: conditional logic for if bucket is emptyAlex Schofield2024-08-201-9/+23
| * feat: revert s3 streaming to previous implementation for uploadingAlex Schofield2024-08-201-31/+25
| * fix(extract_lambda): fix UnboundLocalError when db is called before it is ass...Alex Schofield2024-08-201-0/+1
| * refactor: optimise s3 streaming & file namingAlex Schofield2024-08-191-11/+6
| * feat: use buffers for s3 upload instead of csv filesAlex Schofield2024-08-191-17/+33
| * refactor: remove print statements in process_and_upload_tablesAlex Schofield2024-08-191-2/+0
| * refactor: add timestamp function in extract_lambda.pyAlex Schofield2024-08-191-12/+17
| * chore: reorganise imports in extract_lambdaAlex Schofield2024-08-191-5/+5
| * refactor(retrieve_secrets): add logging for ClientErrorAlex Schofield2024-08-191-0/+1
| * refactor(retrieve_secrets): improve error handling when retrieving secretsAlex Schofield2024-08-191-0/+3
| * refactor(lambda_handler): remove unnecessary else statementAlex Schofield2024-08-191-8/+7
| * refactor(retrieve_secrets): use aws recommended method for retrieving secretsAlex Schofield2024-08-191-8/+11
| * refactor(log): implement logging ancestry - avoid using root loggerAlex Schofield2024-08-191-1/+1
git.ajschof.me — hosted by ajschofield — powered by cgit