aboutsummaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'development' into test/tests_transform_lambdaAlex2024-09-033-105/+225
|\
| * Merge pull request #107 from ajschofield/feat/dataframesbulve-ad2024-08-293-106/+226
| |\ | | | | | | pr: working load lambda when no data is present in the warehouse
| | * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-293-95/+105
| | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 48e7dae according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/107
| | * fix: added comma. Code complete and uploads all tables in one go if no data ↵Ang Bel2024-08-291-10/+9
| | | | | | | | | | | | exists per each table
| | * fix: adds missing dataframes and resolves tables upload to end data ↵Ang Bel2024-08-283-17/+30
| | | | | | | | | | | | warehouse in case the table is empty
| | * wip commitT-Aji2024-08-283-70/+114
| | |
| | * debugging load_lambdaT-Aji2024-08-281-2/+3
| | |
| | * refactored functs to include columns instead of drop columnsT-Aji2024-08-283-57/+110
| | |
* | | wip finished testing the process and upload parquetlian-manonog2024-08-281-2/+4
|/ /
* | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-4/+3
| | | | | | | | | | | | This commit fixes the style issues introduced in 6c85677 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/106
* | chore: get out of merge hellAlex2024-08-281-19/+2
| |
* | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-6/+9
| | | | | | | | | | | | This commit fixes the style issues introduced in 4651e2f according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/106
* | Merge branch 'test/transform-lambda' of ↵lian-manonog2024-08-281-0/+18
|\ \ | |/ |/| | | https://github.com/ajschofield/de-project-bentley into test/transform-lambda
| * style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-22/+18
| | | | | | | | | | | | This commit fixes the style issues introduced in ad357ff according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/105
| * test updates to transform lambda handlerHastarTara2024-08-271-25/+34
| |
| * tests for bucket_name helperHastarTara2024-08-271-6/+11
| |
* | Merge branch 'development' of ↵HastarTara2024-08-282-25/+68
|\ \ | | | | | | | | | https://github.com/ajschofield/de-project-bentley into development
| * | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-59/+59
| | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in d0b0fa9 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/102
| * | Merge branch 'development' into feature/load-lambda-testsEllie2024-08-281-59/+26
| |\ \
| * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-85/+97
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 08c971f according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/102
| * | | add json.loadsEllie2024-08-281-4/+5
| | | |
| * | | amend to inner joinEllie2024-08-281-3/+3
| | | |
| * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-0/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in a05a371 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/98
| * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-282-80/+161
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 1a145a3 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/101
| * | | test: transform refactoring - it now loads parquet files into s3 bucketAng Bel2024-08-282-19/+19
| | | |
| * | | test: refactored fact functions with test passingAng Bel2024-08-281-12/+12
| | | |
| * | | test: fact transformation function for payment test passes, other fact ↵Ang Bel2024-08-281-166/+85
| | | | | | | | | | | | | | | | functions are equivalent, no tests written
| * | | add logger error to lambda handlerEllie2024-08-271-5/+7
| | | |
| * | | wip: update TestLambdaHandler & lambda_handler functionAlex Schofield2024-08-271-8/+11
| | | |
| * | | refactor: add parameter for sm_secretAlex Schofield2024-08-271-3/+6
| | | |
| * | | add json.loads to retrieve secrests functionEllie2024-08-271-6/+9
| | | |
| * | | remove import as not requiredEllie2024-08-271-3/+5
| | | |
* | | | moved extract_l & dataframes into own directory in srcHastarTara2024-08-282-0/+0
| |/ / |/| |
* | | Merge branch 'development' into test/transform-helper-functionsbulve-ad2024-08-271-2/+197
|\ \ \
| * \ \ Merge branch 'development' into test/transform-helper-functionsbulve-ad2024-08-272-2/+198
| |\ \ \
| | * \ \ Merge pull request #98 from ajschofield/test/transform-helper-functionsbulve-ad2024-08-271-57/+54
| | |\ \ \ | | | |/ / | | |/| | pr: refactored facts transformation functions and one test for it
| | | * | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-271-78/+156
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in a05a371 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/98
| | * | | Merge branch 'development' into feature/load-lambdaEllie2024-08-272-5/+413
| | |\ \ \ | | | | |/ | | | |/|
| | * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-231-24/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 0ff2956 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/95
| | * | | add amendendments for upload_dfs_to_dbEllie2024-08-231-13/+34
| | | | |
| | * | | add retrieve secrets functionEllie2024-08-231-1/+20
| | | | |
| | * | | add get transform bucket testEllie2024-08-231-1/+1
| | | | |
| | * | | add amendments to load lambdaEllie2024-08-231-29/+37
| | | | |
| | * | | add testsEllie2024-08-231-3/+58
| | | | |
| | * | | add get transform bucket functionEllie2024-08-231-1/+15
| | | | |
| | * | | add improved convert parquet files to df functionEllie2024-08-231-14/+19
| | | | |
| | * | | add convert parquet to df functionEllie2024-08-221-2/+48
| | | | |
| * | | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-271-78/+152
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 22df92b according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/99
* | | | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-272-80/+161
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 1a145a3 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/101
* | | | | test: transform refactoring - it now loads parquet files into s3 bucketAng Bel2024-08-272-19/+19
|/ / / /
git.ajschof.me — hosted by ajschofield — powered by cgit