index
:
de-project-bentley
development
main
Northcoders project - ETL pipeline with AWS
Alex Schofield
about
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
Commit message (
Expand
)
Author
Age
Files
Lines
*
Merge pull request #84 from ajschofield/feature/transform_lambda
bulve-ad
2024-08-21
4
-4
/
+120
|
\
|
*
Merge branch 'development' into feature/transform_lambda
bulve-ad
2024-08-21
24
-487
/
+676
|
|
\
|
|
/
|
/
|
*
|
Merge pull request #83 from ajschofield/ci/add-dev-tests-yml
Alex
2024-08-20
1
-0
/
+48
|
\
\
|
*
|
ci: remove environment for dev-tests.yml
Alex
2024-08-20
1
-1
/
+0
|
*
|
ci: create dev-tests.yml
Alex
2024-08-20
1
-0
/
+49
|
/
/
*
|
Merge pull request #82 from ajschofield/test/test-branch
Alex
2024-08-20
1
-2
/
+2
|
\
\
|
*
|
test: test trigger for ci/cd
Alex Schofield
2024-08-20
1
-2
/
+2
|
/
/
*
|
Merge pull request #81 from ajschofield/alex/remove-dev-ga-workflow
Alex
2024-08-20
1
-43
/
+0
|
\
\
|
*
|
chore: rm workflow file from development
Alex
2024-08-20
1
-43
/
+0
|
/
/
*
|
Merge pull request #79 from ajschofield/ci/actions-trigger
Alex
2024-08-20
1
-0
/
+1
|
\
\
|
*
|
ci: update deploy.yml
Alex
2024-08-20
1
-0
/
+1
|
/
/
*
|
Merge pull request #78 from ajschofield/main
Alex
2024-08-20
1
-0
/
+3
|
\
\
|
*
\
Merge pull request #77 from ajschofield/alex/lambda-timeout
Alex
2024-08-20
1
-0
/
+3
|
|
\
\
|
|
*
\
Merge branch 'main' into alex/lambda-timeout
Alex
2024-08-20
0
-0
/
+0
|
|
|
\
\
|
|
|
/
/
|
|
/
|
|
|
*
|
|
Merge pull request #76 from ajschofield/development
Alex
2024-08-20
21
-101
/
+1363
|
|
\
\
\
|
|
/
/
/
|
/
|
|
|
|
|
*
|
hotfix: increase lambda timeout
Alex Schofield
2024-08-20
1
-0
/
+3
|
|
/
/
|
/
|
|
*
|
|
Merge pull request #75 from ajschofield/alex/repo-tidy-up
Alex
2024-08-20
3
-129
/
+0
|
\
\
\
|
*
|
|
chore: tidy-up repository & remove unused files
Alex Schofield
2024-08-20
3
-129
/
+0
|
/
/
/
*
|
|
Merge pull request #74 from ajschofield/alex/tf-lambda-changes
Alex
2024-08-20
2
-5
/
+11
|
\
\
\
|
*
|
|
infra(tf): add ListBucket and GetObject permissions
Alex Schofield
2024-08-20
1
-3
/
+6
|
*
|
|
feat: add more logging for debugging
Alex Schofield
2024-08-20
1
-2
/
+5
|
/
/
/
*
|
|
Merge pull request #73 from ajschofield/alex/tf-s3-perms
Alex
2024-08-20
3
-45
/
+63
|
\
\
\
|
*
|
|
infra(tf): force-destroy buckets
Alex Schofield
2024-08-20
1
-1
/
+3
|
*
|
|
refactor: change rows output to debug logger output
Alex Schofield
2024-08-20
1
-2
/
+1
|
*
|
|
feat: add additional logging and exclude unnecessary table
Alex Schofield
2024-08-20
1
-5
/
+7
|
*
|
|
feat: add advanced logging
Alex Schofield
2024-08-20
1
-5
/
+11
|
*
|
|
feat: conditional logic for if bucket is empty
Alex Schofield
2024-08-20
1
-9
/
+23
|
*
|
|
feat: revert s3 streaming to previous implementation for uploading
Alex Schofield
2024-08-20
1
-31
/
+25
|
*
|
|
infra(tf): add missing ListObjectsV2 permission
Alex Schofield
2024-08-20
1
-1
/
+2
*
|
|
|
Merge pull request #72 from ajschofield/feature/test_extract_bucket_helper_fu...
Alex
2024-08-20
1
-2
/
+23
|
\
\
\
\
|
*
|
|
|
style: format code with Autopep8, Black and Ruff Formatter
deepsource-autofix[bot]
2024-08-20
1
-9
/
+5
|
*
|
|
|
Complete: completed testing for extract bucket - all passing
lian-manonog
2024-08-20
1
-2
/
+27
|
/
/
/
/
*
|
|
|
Merge pull request #71 from ajschofield/alex/py-extract-lambda
Alex
2024-08-20
1
-1
/
+1
|
\
\
\
\
|
|
/
/
/
|
/
|
|
|
|
*
|
|
fix(extract_lambda): parse secrets string as json dict to access secret values
Alex Schofield
2024-08-20
1
-1
/
+1
|
/
/
/
*
|
|
Merge pull request #70 from ajschofield/alex/tf-secrets-perms
Alex
2024-08-20
1
-0
/
+27
|
\
\
\
|
*
|
|
infra(tf): add resource access for secrets-manager policy doc
Alex Schofield
2024-08-20
1
-1
/
+1
|
*
|
|
infra(tf): add secrets manager permissions
Alex Schofield
2024-08-20
1
-0
/
+27
|
/
/
/
*
|
|
Merge pull request #69 from ajschofield/alex/py-improvements
Alex
2024-08-20
2
-63
/
+89
|
\
\
\
|
*
|
|
fix(extract_lambda): fix UnboundLocalError when db is called before it is ass...
Alex Schofield
2024-08-20
1
-0
/
+1
|
*
|
|
fix(): update expected response message for second test
Alex Schofield
2024-08-20
1
-1
/
+2
|
*
|
|
chore(tests): rename lambda_handler class test functions
Alex Schofield
2024-08-20
1
-3
/
+3
|
*
|
|
fix(): update expected response message
Alex Schofield
2024-08-20
1
-4
/
+8
|
*
|
|
refactor: optimise s3 streaming & file naming
Alex Schofield
2024-08-19
1
-11
/
+6
|
*
|
|
feat: use buffers for s3 upload instead of csv files
Alex Schofield
2024-08-19
1
-17
/
+33
|
*
|
|
refactor: remove print statements in process_and_upload_tables
Alex Schofield
2024-08-19
1
-2
/
+0
|
*
|
|
refactor: add timestamp function in extract_lambda.py
Alex Schofield
2024-08-19
1
-12
/
+17
|
*
|
|
Merge branch 'development' into alex/py-improvements
Alex Schofield
2024-08-19
1
-7
/
+22
|
|
\
\
\
|
*
|
|
|
chore: reorganise imports in extract_lambda
Alex Schofield
2024-08-19
1
-5
/
+5
|
*
|
|
|
refactor(retrieve_secrets): add logging for ClientError
Alex Schofield
2024-08-19
1
-0
/
+1
|
*
|
|
|
refactor(retrieve_secrets): improve error handling when retrieving secrets
Alex Schofield
2024-08-19
1
-0
/
+3
[next]
git.ajschof.me
— hosted by
ajschofield
— powered by
cgit