Download Latest Version
v2.0.0-beta source code.tar.gz (1.3 MB)
Get an email when there's a new version of Dataproc Templates
| Name | Modified | Size | Downloads / Week |
|---|---|---|---|
| Parent folder | |||
| README.md | 2023-08-07 | 13.0 kB | |
| v0.4.0-beta source code.tar.gz | 2023-08-07 | 1.2 MB | |
| v0.4.0-beta source code.zip | 2023-08-07 | 1.6 MB | |
| Totals: 3 Items | 2.8 MB | 0 | |
0.4.0-beta (2023-08-07)
Features
- 756 postgre sql to big query (#805) (121bf93)
- add bq_dataset_region parameter (a9647ad)
- Add ORACLE_SCHEMA to parameterize scripts (969e314)
- added init file (3273fe7)
- Added Paramterized script file (2951e3a)
- HiveToBiqquery parameterize script (2eec146)
- include logging (ca1f9be)
- include logging (2714b7f)
- include logging (cf0a81c)
- service account as env variable (9c93710)
- service account as env variable (57292f9)
- update nb constants & script name (cb07530)
- update nb with IS_PARAMETERIZED flag (b4dcd35)
- update notebook constants (5c9bdc2)
- update working dir (ea6a52b)
- update working dir for parameterization (5840c88)
- Updated notebook to handle parameters (08dd1c4)
- updated readme for parametrized script (82ff6ad)
- updated readme for parametrized script (afac2a6)
- updated run_notebook (c84883f)
Bug Fixes
- add log_output (09dc88a)
- black formatting (fcab888)
- correct pip install cmd (f9c17c5)
- correct project id variable (5c61b94)
- correct sqlachemy execute (14cb3d3)
- Deal with upgraded google_cloud_pipeline_components.experimental vs v1 (4551fab)
- Deal with upgraded google_cloud_pipeline_components.experimental vs v1 (2304179)
- deletes gcs folder created after all tests run (66e3000)
- except statement (b88758a)
- Fix merge conflict (deb3a43)
- Fix merge from main (5a75ae0)
- Fix merge from main (6ad8fe6)
- Fix merge from main (2dd4be5)
- Fix merge from main (fcc8ca6)
- Fix merge from main (90a65ec)
- ignored some tests in general template (ae74321)
- Implement de-duplication (b1c9f8a)
- implement de-duplication changes (1aeef92)
- implement de-duplication changes (02c2508)
- implement get_common_args function (6525dc5)
- implement get_env_vars in base script (0734fb7)
- integration test - change custom container image (#810) (8943508)
- make subnet optional variable (64c0e68)
- make subnet optional variable (294eea5)
- modified except block (8871b25)
- move max_parallelism from common_args (1f88912)
- Notebook MYSQLTABLE_LIST to MYSQL_TABLE_LIST (5abe159)
- Notebook MYSQLTABLE_LIST to MYSQL_TABLE_LIST (e22fb72)
- oracle table list parsing corrected (5f33fb4)
- oracle table list parsing corrected (7f1e913)
- raise exception on job failure (e5b1a99)
- remove common args from script (fc125f2)
- remove duplicate get_env_var (73bad48)
- remove log_level from nb_parameters (634d926)
- remove log_level from nb_parameters (fe49799)
- remove project id constant (e9173bb)
- removed duplicate arugment (e0c3262)
- removed pubsub output project parameter (24a9d18)
- removed pubsub output project parameter (838c23a)
- removed unused outputProjectID (5f106c4)
- removing version parameter while running with cluster (d9faf34)
- renamed init file (aa60431)
- Resolve conflicting changes (8ab7e0d)
- small fixes in notebook (155c5c8)
- update project id variable (97e58f1)
- update PROJECT_ID to PROJECT (1a06e52)
- update scipt_name with oracletopostgres (5dac5d4)
- update spark to 3.3.2 (60757dd)
- updated indentation (306e4e5)
- updated log_output (0b2a510)
- updated notebook with tag parameters (df007b4)
- updated notebook_constants (c4aaaa7)
- updated pom.xml to avoid commons-cli conflict in cluster mode (40cc8f7)
- updated postgres arg in notebook_constants (5900066)
- updated sql connection (8c5680d)
- updated sql query error for boolean value (a71e7f1)
- updated to UUID GCS staging location (a6fef9e)
