aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* infra(tf): restore make_layer_zip & add layer_01/layer_02 creation in same ↵Alex Schofield2024-08-293-17/+17
| | | | script
* chore: update requirements file names in bash scriptsAlex Schofield2024-08-292-2/+2
|
* chore: add new requirements.txt files for extract/transform/lambda layersAlex Schofield2024-08-292-0/+12
|
* refactor: separate layer zip prepAlex Schofield2024-08-292-10/+10
|
* chore: rm old make_layer_zip.sh scriptAlex Schofield2024-08-291-8/+0
|
* feat: separate layer zip generation for lambda functionsAlex Schofield2024-08-292-0/+17
|
* chore: remove car_data.parquetAlex Schofield2024-08-291-0/+0
|
* infra(tf): update load function_name & lambda_function_arnAlex Schofield2024-08-291-2/+2
|
* Merge pull request #111 from ajschofield/alex/tf-layer-fixAlex2024-08-291-2/+2
|\ | | | | pr: fix layer paths
| * infra(tf): fix layer pathsAlex Schofield2024-08-291-2/+2
|/
* Merge pull request #110 from ajschofield/a-e/load_lambdaAlex2024-08-291-29/+64
|\ | | | | pr: update tests for load_lambda
| * test: update upload_dfs_to_database testsAlex Schofield2024-08-281-28/+24
| |
| * test: add staticmethod decorator to remaining methodsAlex Schofield2024-08-281-16/+24
| |
| * chore: remove commented codeAlex Schofield2024-08-281-5/+0
| |
| * wip: add tests for upload_dfs_to_databaseAlex Schofield2024-08-281-4/+31
| |
| * test: add staticmethod decorator to methodsAlex Schofield2024-08-281-3/+6
| |
| * test: re-add json module importAlex Schofield2024-08-281-0/+1
| |
| * test: import load_lambda functions individuallyAlex Schofield2024-08-281-1/+8
| |
| * test: remove unnecessary importsAlex Schofield2024-08-281-2/+0
| |
| * test: remove aws_credentials parameter from fixturesAlex Schofield2024-08-281-2/+2
| |
* | Merge pull request #109 from ajschofield/mainAlex2024-08-291-0/+42
|\ \ | | | | | | pr: merge main changes back into development
| * | ci: create deploy.ymlAlex2024-08-291-0/+42
| | |
| * | Merge pull request #108 from ajschofield/developmentAlex2024-08-2918-89/+1671
| |\ \ | |/ / |/| | pr: final push, data warehouse is currently empty to test that it uploads through terraform
* | | Merge pull request #107 from ajschofield/feat/dataframesbulve-ad2024-08-296-109/+233
|\ \ \ | | | | | | | | pr: working load lambda when no data is present in the warehouse
| * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-293-95/+105
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 48e7dae according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/107
| * | | fix: added comma. Code complete and uploads all tables in one go if no data ↵Ang Bel2024-08-291-10/+9
| | | | | | | | | | | | | | | | exists per each table
| * | | fix: adds missing dataframes and resolves tables upload to end data ↵Ang Bel2024-08-284-18/+35
| | | | | | | | | | | | | | | | warehouse in case the table is empty
| * | | wip commitT-Aji2024-08-284-71/+115
| | | |
| * | | debugging load_lambdaT-Aji2024-08-281-2/+3
| | | |
| * | | refactored functs to include columns instead of drop columnsT-Aji2024-08-284-58/+111
| | | |
* | | | Merge pull request #106 from ajschofield/test/transform-lambdaAlex2024-08-283-18/+71
|\ \ \ \ | |_|_|/ |/| | | PR: tests for transform into dev
| * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-4/+3
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 6c85677 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/106
| * | | chore: get out of merge hellAlex2024-08-281-19/+2
| | | |
| * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-281-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 4651e2f according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/106
| * | | Merge branch 'test/transform-lambda' of ↵lian-manonog2024-08-282-36/+55
| |\ \ \ | | | | | | | | | | | | | | | https://github.com/ajschofield/de-project-bentley into test/transform-lambda
| | * | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-282-58/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in ad357ff according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/105
| * | | | Merge branch 'development' of ↵lian-manonog2024-08-287-111/+593
| |\ \ \ \ | |/ / / / |/| / / / | |/ / / https://github.com/ajschofield/de-project-bentley into test/transform-lambda
* | | | Merge pull request #104 from ajschofield/temp_branch_for_prHastarTara2024-08-281-2/+4
|\ \ \ \ | |_|/ / |/| | | Temp branch for pr
| * | | work on lambda handler dirctory config 2HastarTara2024-08-281-8/+3
| | | |
| * | | work on lambda handler dirctory configHastarTara2024-08-281-0/+7
| | | |
* | | | Merge pull request #103 from ajschofield/temp_branch_for_prHastarTara2024-08-282-0/+0
|\| | | | | | | | | | | Temp branch for pr
| * | | Merge branch 'development' of ↵HastarTara2024-08-284-45/+206
| |\ \ \ | |/ / / |/| | | | | | | https://github.com/ajschofield/de-project-bentley into development
* | | | Merge pull request #102 from ajschofield/feature/load-lambda-testsEllie2024-08-284-45/+206
|\ \ \ \ | | | | | | | | | | pr: feature/load lambda tests
| * | | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-282-61/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in d0b0fa9 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/102
| * | | | Merge branch 'development' into feature/load-lambda-testsEllie2024-08-282-72/+33
| |\ \ \ \ | |/ / / / |/| | | |
| * | | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-283-104/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in 08c971f according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/102
| * | | | add json.loads to retrieve secretsEllie2024-08-281-2/+2
| | | | |
| * | | | add json.loadsEllie2024-08-281-4/+5
| | | | |
| * | | | amend to inner joinEllie2024-08-281-3/+3
| | | | |
| * | | | style: format code with Autopep8, Black and Ruff Formatterdeepsource-autofix[bot]2024-08-282-0/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes the style issues introduced in a05a371 according to the output from Autopep8, Black and Ruff Formatter. Details: https://github.com/ajschofield/de-project-bentley/pull/98
git.ajschof.me — hosted by ajschofield — powered by cgit