aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'test/test_transform_lambda' into test/dataframesbulve-ad2024-08-2311-283/+340
|\
| * wip: writing more tests for the helper functionslian-manonog2024-08-231-2/+12
| |
| * wip: pushing againlian-manonog2024-08-233-9/+47
| |
| * Merge pull request #94 from ajschofield/feature/transform-fact-sales-orderlian-manonog2024-08-235-10/+611
| |\ | | | | | | PR: rebased the fact-sales branch with the datafrrames branch - now merging into development
| | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-4/+3
| | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 88f1ef7 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/94
| | * Removed redundant empty lines of codelian-manonog2024-08-231-4/+0
| | |
| | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-232-10/+15
| | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 3ff2182 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/93
| | * trying to resolce git index issue conflicts - commiting was the only solutionlian-manonog2024-08-232-9/+8
| | |
| | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-234-180/+319
| | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 8e20c5c according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/93
| | * Deleted the fact_table schema py fileslian-manonog2024-08-234-233/+157
| | | | | | | | | | | | | | | | | | Completed Lambda_handler for transform_lambda - and other helper functions. Testing is still to be done. Need to implement lambda layer to share helper functions across all lambdas
| * | Merge pull request #90 from ajschofield/alex/fix-extract-lambda-testAlex2024-08-224-34/+90
| |\ \ | | | | | | | | pr: fix extract lambda tests
| | * | fix: improve error handling for list_existing_s3_files and testsAlex Schofield2024-08-222-8/+18
| | | |
| | * | fix: handle no buckets properlyAlex Schofield2024-08-222-5/+8
| | | |
| | * | chore: skip transform_lambda tests are they are brokenAlex Schofield2024-08-221-0/+2
| | | |
| | * | chore: skip secrets_manager tests are they are brokenAlex Schofield2024-08-221-0/+4
| | | |
| | * | fix: resolve import errorAlex Schofield2024-08-221-1/+1
| | | |
| | * | fix: mock aws services before importing src functionsAlex Schofield2024-08-221-9/+11
| | | |
| | * | fix: change default parameters for bucket_name and clientAlex Schofield2024-08-221-2/+9
| | | |
| | * | fix: inject mock_config into interface error testAlex Schofield2024-08-221-2/+15
| | | |
| | * | fix: convert credentials to json dictAlex Schofield2024-08-221-7/+9
| | | |
| | * | fix: add missing mock_conn fixtureAlex Schofield2024-08-221-0/+7
| | | |
| | * | fix: remove bucket deletion for index error testAlex Schofield2024-08-221-3/+2
| | | |
| | * | fix: create two mocked buckets and check if extract_bucket is returnedAlex Schofield2024-08-221-1/+8
| | | |
| | * | feat: autouse credentials fixtureAlex Schofield2024-08-221-1/+1
| | | |
| | * | fix: update additional fixtures to use scope functionAlex Schofield2024-08-221-2/+2
| | | |
| | * | fix: change fixture scope to function instead of classAlex Schofield2024-08-221-2/+2
| | | |
| * | | Merge pull request #92 from ajschofield/alex/ci-artifact-uploadAlex2024-08-221-0/+7
| |\ \ \ | | | | | | | | | | pr: upload coverage report as artifact
| | * | | ci: upload coverage report as artifactAlex2024-08-221-0/+7
| |/ / /
| * | | Merge pull request #91 from ajschofield/alex/ci-pytest-cov-reportAlex2024-08-221-2/+2
| |\ \ \ | | |/ / | |/| | pr: simplify pytest output & add coverage report
| | * | ci: simplify pytest output & add coverage reportAlex2024-08-221-2/+2
| |/ /
| * | Merge pull request #89 from ajschofield/alex/re-add-sm-testAlex2024-08-221-0/+84
| |\ \ | | | | | | | | pr: re-add test_secrets_manager
| | * | test: re-add test_secrets_managerAlex2024-08-221-0/+84
| |/ / | | | | | | Amended import path to extract_lambda
| * | Merge pull request #88 from ajschofield/alex/rm-sm-testAlex2024-08-211-84/+0
| |\ \ | | | | | | | | pr: delete test for secrets_manager.py
| | * | chore: delete test for secrets_manager.pyAlex2024-08-211-84/+0
| |/ / | | | | | | We don't need this anymore since we removed the secrets_manager.py file, and it's making the GH Action fail too.
| * | Merge pull request #87 from ajschofield/ci/add-pythonpathAlex2024-08-211-0/+3
| |\ \ | | | | | | | | pr: update dev-tests.yml
| | * | ci: update dev-tests.ymlAlex2024-08-211-0/+3
| |/ /
| * | Merge pull request #86 from ajschofield/alex/fix-ciAlex2024-08-211-0/+1
| |\ \ | | | | | | | | pr: update dev-tests.yml
| | * | Update dev-tests.ymlAlex2024-08-211-0/+1
| |/ /
* | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-232-155/+330
| | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 821e241 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/96
* | | tests: additional tests written (pass) for dim tables transformation. Fact ↵Ang Bel2024-08-232-61/+82
| | | | | | | | | | | | transformation functions not yet tested
* | | fix: refactoring for create_dim_date to include all date columns from all ↵Ang Bel2024-08-231-20/+13
| | | | | | | | | | | | fact dfs, tested on dummy data. Tests are not written
* | | removed duplicate functionsT-Aji2024-08-231-16/+69
| | |
* | | removed duplicate functionsT-Aji2024-08-231-89/+28
| |/ |/|
* | dataframes combined into one fileT-Aji2024-08-231-0/+238
| |
* | Merge branch 'feature/transform-fact-sales-order' of ↵T-Aji2024-08-234-10/+62
|\ \ | | | | | | | | | https://github.com/ajschofield/de-project-bentley into feature/transform-fact-sales-order
| * | wip: wrote pseudocode for lambda handler in writing df to parquet file ↵lian-manonog2024-08-223-6/+56
| | | | | | | | | | | | format and uploading the parquet files
| * | Refactored tests for transform lambda - all passing nowlian-manonog2024-08-221-4/+6
| | |
* | | payment schema addedT-Aji2024-08-232-2/+46
|/ /
* | complete code for tables for purchase schema including a scrape for ↵Ang Bel2024-08-221-18/+48
| | | | | | | | currency table. Test to be done
* | add fact tableT-Aji2024-08-221-50/+0
| |
git.ajschof.me — hosted by ajschofield — powered by cgit