index
:
de-project-bentley
development
main
Northcoders project - ETL pipeline with AWS
Alex Schofield
about
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
Commit message (
Expand
)
Author
Age
Files
Lines
*
Update README.md
HEAD
main
Alex
2024-09-03
1
-1
/
+1
*
Update README.md
Alex
2024-09-03
1
-1
/
+56
*
Merge pull request #117 from ajschofield/development
Alex
2024-09-03
9
-66
/
+223
|
\
|
*
Merge pull request #116 from ajschofield/test/tests_transform_lambda
development
Alex
2024-09-03
3
-29
/
+146
|
|
\
|
|
*
Merge branch 'development' into test/tests_transform_lambda
Alex
2024-09-03
14
-191
/
+422
|
|
|
\
|
|
|
/
|
|
/
|
|
*
|
Merge pull request #115 from ajschofield/a-a/fix-tf-again
Alex
2024-09-03
6
-37
/
+77
|
|
\
\
|
|
*
|
infra(tf)/wip: update lambda layer creation scripting
Alex Schofield
2024-08-29
1
-31
/
+50
|
|
*
|
infra(tf): restore make_layer_zip & add layer_01/layer_02 creation in same sc...
Alex Schofield
2024-08-29
3
-17
/
+17
|
|
*
|
chore: update requirements file names in bash scripts
Alex Schofield
2024-08-29
2
-2
/
+2
|
|
*
|
chore: add new requirements.txt files for extract/transform/lambda layers
Alex Schofield
2024-08-29
2
-0
/
+12
|
|
*
|
refactor: separate layer zip prep
Alex Schofield
2024-08-29
2
-10
/
+10
|
|
*
|
chore: rm old make_layer_zip.sh script
Alex Schofield
2024-08-29
1
-8
/
+0
|
|
*
|
feat: separate layer zip generation for lambda functions
Alex Schofield
2024-08-29
2
-0
/
+17
|
|
*
|
chore: remove car_data.parquet
Alex Schofield
2024-08-29
1
-0
/
+0
|
|
*
|
infra(tf): update load function_name & lambda_function_arn
Alex Schofield
2024-08-29
1
-2
/
+2
|
|
|
*
wip: lambdahandler tests fail - unexpected errors
lian-manonog
2024-08-29
2
-2
/
+5
|
|
|
*
wip: lambda handler tests not working - suspecting its something in dataframes
lian-manonog
2024-08-28
1
-10
/
+17
|
|
|
*
wip: testing lambda handler. Added Dict of Df patch fixture
lian-manonog
2024-08-28
1
-16
/
+50
|
|
|
*
wip: completed and added to testprocesstoparquetuploads3
lian-manonog
2024-08-28
1
-11
/
+19
|
|
|
*
wip finished testing the process and upload parquet
lian-manonog
2024-08-28
2
-7
/
+60
|
|
|
*
wip: testing the process helper function
lian-manonog
2024-08-28
1
-10
/
+26
*
|
|
|
Merge pull request #114 from ajschofield/development
Alex
2024-08-30
1
-13
/
+7
|
\
|
|
|
|
*
|
|
Merge pull request #113 from ajschofield/alex/readme-update
Alex
2024-08-30
1
-13
/
+7
|
|
\
\
\
|
|
|
/
/
|
|
/
|
|
|
|
*
|
Update README.md
Alex
2024-08-30
1
-13
/
+7
|
|
/
/
*
|
|
Merge pull request #112 from ajschofield/development
Alex
2024-08-29
2
-31
/
+66
|
\
|
|
|
*
|
Merge pull request #111 from ajschofield/alex/tf-layer-fix
Alex
2024-08-29
1
-2
/
+2
|
|
\
\
|
|
*
|
infra(tf): fix layer paths
Alex Schofield
2024-08-29
1
-2
/
+2
|
|
/
/
|
*
|
Merge pull request #110 from ajschofield/a-e/load_lambda
Alex
2024-08-29
1
-29
/
+64
|
|
\
\
|
|
*
|
test: update upload_dfs_to_database tests
Alex Schofield
2024-08-28
1
-28
/
+24
|
|
*
|
test: add staticmethod decorator to remaining methods
Alex Schofield
2024-08-28
1
-16
/
+24
|
|
*
|
chore: remove commented code
Alex Schofield
2024-08-28
1
-5
/
+0
|
|
*
|
wip: add tests for upload_dfs_to_database
Alex Schofield
2024-08-28
1
-4
/
+31
|
|
*
|
test: add staticmethod decorator to methods
Alex Schofield
2024-08-28
1
-3
/
+6
|
|
*
|
test: re-add json module import
Alex Schofield
2024-08-28
1
-0
/
+1
|
|
*
|
test: import load_lambda functions individually
Alex Schofield
2024-08-28
1
-1
/
+8
|
|
*
|
test: remove unnecessary imports
Alex Schofield
2024-08-28
1
-2
/
+0
|
|
*
|
test: remove aws_credentials parameter from fixtures
Alex Schofield
2024-08-28
1
-2
/
+2
|
*
|
|
Merge pull request #109 from ajschofield/main
Alex
2024-08-29
1
-0
/
+42
|
|
\
\
\
|
|
/
/
/
|
/
|
|
|
*
|
|
|
ci: create deploy.yml
Alex
2024-08-29
1
-0
/
+42
*
|
|
|
Merge pull request #108 from ajschofield/development
Alex
2024-08-29
18
-89
/
+1671
|
\
|
|
|
|
*
|
|
Merge pull request #107 from ajschofield/feat/dataframes
bulve-ad
2024-08-29
6
-109
/
+233
|
|
\
\
\
|
|
|
_
|
/
|
|
/
|
|
|
|
*
|
style: format code with Autopep8, Black and Ruff Formatter
deepsource-autofix[bot]
2024-08-29
3
-95
/
+105
|
|
*
|
fix: added comma. Code complete and uploads all tables in one go if no data e...
Ang Bel
2024-08-29
1
-10
/
+9
|
|
*
|
fix: adds missing dataframes and resolves tables upload to end data warehouse...
Ang Bel
2024-08-28
4
-18
/
+35
|
|
*
|
wip commit
T-Aji
2024-08-28
4
-71
/
+115
|
|
*
|
debugging load_lambda
T-Aji
2024-08-28
1
-2
/
+3
|
|
*
|
refactored functs to include columns instead of drop columns
T-Aji
2024-08-28
4
-58
/
+111
|
*
|
|
Merge pull request #106 from ajschofield/test/transform-lambda
Alex
2024-08-28
3
-18
/
+71
|
|
\
\
\
|
|
|
_
|
/
|
|
/
|
|
|
|
*
|
style: format code with Autopep8, Black and Ruff Formatter
deepsource-autofix[bot]
2024-08-28
1
-4
/
+3
|
|
*
|
chore: get out of merge hell
Alex
2024-08-28
1
-19
/
+2
[next]
git.ajschof.me
— hosted by
ajschofield
— powered by
cgit