aboutsummaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* amend to inner joinEllie2024-08-281-3/+3
|
* style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-0/+50
| | | | | | | This commit fixes the style issues introduced in a05a371 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/98
* style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-282-80/+161
| | | | | | This commit fixes the style issues introduced in 1a145a3 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/101
* test: transform refactoring - it now loads parquet files into s3 bucketAng Bel2024-08-282-19/+19
|
* test: refactored fact functions with test passingAng Bel2024-08-281-12/+12
|
* test: fact transformation function for payment test passes, other fact ↵Ang Bel2024-08-281-166/+85
| | | | functions are equivalent, no tests written
* add logger error to lambda handlerEllie2024-08-271-5/+7
|
* wip: update TestLambdaHandler & lambda_handler functionAlex Schofield2024-08-271-8/+11
|
* refactor: add parameter for sm_secretAlex Schofield2024-08-271-3/+6
|
* add json.loads to retrieve secrests functionEllie2024-08-271-6/+9
|
* remove import as not requiredEllie2024-08-271-3/+5
|
* Merge branch 'development' into feature/load-lambdaEllie2024-08-272-5/+413
|\
| * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-272-41/+39
| | | | | | | | | | | | This commit fixes the style issues introduced in c68f63f according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/97
| * wip: refactored fact payment functionlian-manonog2024-08-272-8/+9
| |
| * wip: added a bracket in dataframeslian-manonog2024-08-272-3/+4
| |
| * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-35/+41
| | | | | | | | | | | | This commit fixes the style issues introduced in 8f75a47 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/96
| * Merge branch 'test/test_transform_lambda' into test/dataframesbulve-ad2024-08-236-248/+194
| |\
| | * wip: pushing againlian-manonog2024-08-232-6/+7
| | |
| | * Merge pull request #94 from ajschofield/feature/transform-fact-sales-orderlian-manonog2024-08-232-5/+484
| | |\ | | | | | | | | PR: rebased the fact-sales branch with the datafrrames branch - now merging into development
| | | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 3ff2182 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/93
| | | * trying to resolce git index issue conflicts - commiting was the only solutionlian-manonog2024-08-231-9/+4
| | | |
| | | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-232-157/+236
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 8e20c5c according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/93
| | | * Deleted the fact_table schema py fileslian-manonog2024-08-234-233/+157
| | | | | | | | | | | | | | | | | | | | | | | | Completed Lambda_handler for transform_lambda - and other helper functions. Testing is still to be done. Need to implement lambda layer to share helper functions across all lambdas
| * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-96/+154
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 821e241 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/96
| * | | tests: additional tests written (pass) for dim tables transformation. Fact ↵Ang Bel2024-08-231-14/+16
| | | | | | | | | | | | | | | | transformation functions not yet tested
| * | | fix: refactoring for create_dim_date to include all date columns from all ↵Ang Bel2024-08-231-20/+13
| | | | | | | | | | | | | | | | fact dfs, tested on dummy data. Tests are not written
| * | | removed duplicate functionsT-Aji2024-08-231-89/+28
| | |/ | |/|
| * | dataframes combined into one fileT-Aji2024-08-231-0/+238
| | |
| * | Merge branch 'feature/transform-fact-sales-order' of ↵T-Aji2024-08-232-5/+53
| |\ \ | | | | | | | | | | | | https://github.com/ajschofield/de-project-bentley into feature/transform-fact-sales-order
| | * | wip: wrote pseudocode for lambda handler in writing df to parquet file ↵lian-manonog2024-08-222-5/+53
| | | | | | | | | | | | | | | | format and uploading the parquet files
| * | | payment schema addedT-Aji2024-08-232-2/+46
| |/ /
| * | complete code for tables for purchase schema including a scrape for ↵Ang Bel2024-08-221-18/+48
| | | | | | | | | | | | currency table. Test to be done
| * | add fact tableT-Aji2024-08-221-50/+0
| | |
| * | Merge branch 'feature/transform-fact-sales-order' of ↵T-Aji2024-08-221-23/+30
| |\ \ | | | | | | | | | | | | https://github.com/ajschofield/de-project-bentley into feature/transform-fact-sales-order
| | * | refactored approach to writing transformation as functions per df. WIPAng Bel2024-08-221-23/+30
| | | |
| * | | add fact tableT-Aji2024-08-221-4/+31
| |/ /
| * | passing tests create_dim_design and create_dim_staffT-Aji2024-08-221-0/+50
| | |
| * | refactor df creation into funcT-Aji2024-08-221-70/+34
| | |
| * | wip: creating dim dataframesT-Aji2024-08-223-9/+58
| |\ \
| | * | wip: fact_purchase_order schemalian-manonog2024-08-213-3/+37
| | | |
| | * | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-211-18/+68
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 20a3bd8 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/85
| * | | began dim_date dfT-Aji2024-08-211-18/+17
| |/ /
| * | 5 dim tables createdT-Aji2024-08-211-15/+33
| | |
| * | add dim tables: design, staff, currency, location (wip)Ellie2024-08-211-0/+54
| | |
* | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-24/+51
| | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 0ff2956 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/95
* | | add amendendments for upload_dfs_to_dbEllie2024-08-231-13/+34
| | |
* | | add retrieve secrets functionEllie2024-08-231-1/+20
| | |
* | | add get transform bucket testEllie2024-08-231-1/+1
| | |
* | | add amendments to load lambdaEllie2024-08-231-29/+37
| | |
* | | add testsEllie2024-08-231-3/+58
| | |
git.ajschof.me — hosted by ajschofield — powered by cgit