Apache Iceberg Files

What's Changed

• Build: Bump software.amazon.awssdk:bom from 2.30.11 to 2.30.16 by @dependabot in https://github.com/apache/iceberg/pull/12208
• Build: Bump com.google.cloud:libraries-bom from 26.53.0 to 26.54.0 by @dependabot in https://github.com/apache/iceberg/pull/12207
• Build: Bump org.xerial:sqlite-jdbc from 3.48.0.0 to 3.49.0.0 by @dependabot in https://github.com/apache/iceberg/pull/12206
• Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4.1 to 5.4.2 by @dependabot in https://github.com/apache/iceberg/pull/12209
• Docs: Fix expire_snapshots output by @manuzhang in https://github.com/apache/iceberg/pull/12213
• Update LICENSE/NOTICE files with last dependency updates by @jbonofre in https://github.com/apache/iceberg/pull/12214
• Doc: update note about notebook in spark-quickstart by @xxchan in https://github.com/apache/iceberg/pull/11996
• Spec: Support geo type by @szehon-ho in https://github.com/apache/iceberg/pull/10981
• Build: skip scheduled docker image publish workflows on forks by @dramaticlly in https://github.com/apache/iceberg/pull/12218
• Docs: Add missing types to the spec v3 summary by @wgtmac in https://github.com/apache/iceberg/pull/12219
• Spec: Typo - missing be by @RussellSpitzer in https://github.com/apache/iceberg/pull/12229
• Docs: Site update for 1.8.0 release by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12242
• spec: Remove source-ids for V{1,2} tables by @Fokko in https://github.com/apache/iceberg/pull/12161
• Build: Update revAPI to compare against 1.8.0 by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12244
• Update release version to 1.8.0 in doap.rdf by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12247
• Infra: Update Iceberg bug report template for 1.8.0 by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12248
• Docs: Fix formatting of 1.8.0 release notes by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12249
• Core: Adjust Jackson settings to handle large metadata json by @bryanck in https://github.com/apache/iceberg/pull/12224
• Build: Bump Hive to 2.3.10 by @nastra in https://github.com/apache/iceberg/pull/12253
• Build: Clean up dependencies by @nastra in https://github.com/apache/iceberg/pull/12252
• Core: Add InternalData read and write builders by @rdblue in https://github.com/apache/iceberg/pull/12060
• API, Core: Support default values in UpdateSchema by @rdblue in https://github.com/apache/iceberg/pull/12211
• update site to include iceberg summit link by @danicafine in https://github.com/apache/iceberg/pull/12256
• Core: Fix divide by zero when adjust split size by @hantangwangd in https://github.com/apache/iceberg/pull/12201
• Spark: Remove unused PruneColumnsWithReordering class by @rdblue in https://github.com/apache/iceberg/pull/12258
• Spark: Fix assertion checks by @nastra in https://github.com/apache/iceberg/pull/12255
• API: Deprecate NestedType.of in favor of builder by @rdblue in https://github.com/apache/iceberg/pull/12227
• Docker: Pin QEMU version temporarily by @ebyhr in https://github.com/apache/iceberg/pull/12262
• OpenAPI: Add RemoveSchemas REST update type by @gaborkaszab in https://github.com/apache/iceberg/pull/12022
• Minor: update Learn More to point to spark quickstart by @danicafine in https://github.com/apache/iceberg/pull/12272
• Build: Bump datamodel-code-generator from 0.27.2 to 0.28.1 by @manuzhang in https://github.com/apache/iceberg/pull/12290
• Spark 3.5: Fix job description of RewriteTablePathSparkAction by @ebyhr in https://github.com/apache/iceberg/pull/12282
• Build: Bump io.netty:netty-buffer from 4.1.117.Final to 4.1.118.Final by @dependabot in https://github.com/apache/iceberg/pull/12287
• Build: Bump software.amazon.awssdk:bom from 2.30.16 to 2.30.21 by @dependabot in https://github.com/apache/iceberg/pull/12286
• OpenAPI: Add overwrite option when registering an iceberg table by @dramaticlly in https://github.com/apache/iceberg/pull/12239
• Build: Bump mkdocs-material from 9.6.3 to 9.6.4 by @dependabot in https://github.com/apache/iceberg/pull/12284
• Core: Fix non-setting row-lineage from table properties on initial table creation by @tomtongue in https://github.com/apache/iceberg/pull/12307
• API: Reject unknown type for required fields and validate defaults by @rdblue in https://github.com/apache/iceberg/pull/12302
• API: Fix TestInclusiveMetricsEvaluator notStartsWith tests by @rdblue in https://github.com/apache/iceberg/pull/12303
• Core: add variant type support by @aihuaxu in https://github.com/apache/iceberg/pull/11831
• Fix CI: Update tests with UnknownType from Required to Optional by @Fokko in https://github.com/apache/iceberg/pull/12316
• Docs: Refactor site navigation bar by @manuzhang in https://github.com/apache/iceberg/pull/12289
• Parquet: Implement Variant readers by @rdblue in https://github.com/apache/iceberg/pull/12139
• Docs: Add rewrite-table-path in spark procedure by @dramaticlly in https://github.com/apache/iceberg/pull/12115
• Parquet: Fix errorprone warning by @ebyhr in https://github.com/apache/iceberg/pull/12324
• Docs: add apache amoro(incubating) with iceberg (#11965) by @czy006 in https://github.com/apache/iceberg/pull/11966
• Parquet: Fix performance regression in reader init by @bryanck in https://github.com/apache/iceberg/pull/12305
• Core: Fallback to GET requests for namespace/table/view exists checks by @nastra in https://github.com/apache/iceberg/pull/12314
• Fix: fix apache amoro ams doc pic ref by @czy006 in https://github.com/apache/iceberg/pull/12332
• Revert "Core: Serialize null when there is no current snapshot" by @Fokko in https://github.com/apache/iceberg/pull/12312
• [1.8.x] Parquet: Fix performance regression in reader init (#12305) by @nastra in https://github.com/apache/iceberg/pull/12329
• Checkstyle: Apply the same generic type naming rules to interfaces and classes by @pvary in https://github.com/apache/iceberg/pull/12333
• Kafka: Pin Kafka-Connect version to fix integration tests by @nastra in https://github.com/apache/iceberg/pull/12340
• Docs: Fix link of catalog in terms.md by @wangyinsheng in https://github.com/apache/iceberg/pull/12326
• Docs: Add documentation for Rate limiting in Spark Structured Streaming by @singhpk234 in https://github.com/apache/iceberg/pull/12217
• Spark 3.5: Fix Incorrect Spec Used With AddFiles Procedure by @RussellSpitzer in https://github.com/apache/iceberg/pull/12319
• Parquet: Remove deprecated VectorizedReader.setRowGroupInfo and ParquetValueReader.setPageSource by @ebyhr in https://github.com/apache/iceberg/pull/12321
• Bump versions in {LICENSE,NOTICE} by @Fokko in https://github.com/apache/iceberg/pull/12337
• Spark: Remove Spark 3.3 support by @manuzhang in https://github.com/apache/iceberg/pull/12279
• Core: Remove deprecated Util.blockLocations method and StructCopy class by @ebyhr in https://github.com/apache/iceberg/pull/12320
• Core: Handle partition evolution case in PartitionStatsUtil#computeStats by @deniskuzZ in https://github.com/apache/iceberg/pull/12137
• Core, Spark: Remove deprecated code for 1.9.0 by @nastra in https://github.com/apache/iceberg/pull/12336
• API: Move variant to API and add extract expression by @rdblue in https://github.com/apache/iceberg/pull/12304
• Core: Remove namespace/table/view HEAD endpoints from defaults by @nastra in https://github.com/apache/iceberg/pull/12351
• Nit: Remove additional 'Iceberg' in Puffin footer payload by @tomtongue in https://github.com/apache/iceberg/pull/12369
• API: Move Variant interfaces and serialized implementations to API by @rdblue in https://github.com/apache/iceberg/pull/12374
• Flink: Fix the comment error in SketchDataStatistics by @Guosmilesmile in https://github.com/apache/iceberg/pull/12375
• Core: Don't remove trailing slash from absolute paths by @adutra in https://github.com/apache/iceberg/pull/12389
• Build: Bump mkdocs-material from 9.6.4 to 9.6.5 by @dependabot in https://github.com/apache/iceberg/pull/12386
• Build: Bump org.xerial:sqlite-jdbc from 3.49.0.0 to 3.49.1.0 by @dependabot in https://github.com/apache/iceberg/pull/12385
• Build: Bump org.awaitility:awaitility from 4.2.2 to 4.3.0 by @dependabot in https://github.com/apache/iceberg/pull/12384
• Build: Bump nessie from 0.102.5 to 0.103.0 by @dependabot in https://github.com/apache/iceberg/pull/12383
• Build: Bump com.google.cloud:libraries-bom from 26.54.0 to 26.55.0 by @dependabot in https://github.com/apache/iceberg/pull/12382
• Build: Bump testcontainers from 1.20.4 to 1.20.5 by @dependabot in https://github.com/apache/iceberg/pull/12380
• Build: Bump software.amazon.awssdk:bom from 2.30.21 to 2.30.26 by @dependabot in https://github.com/apache/iceberg/pull/12379
• Spec: Allow Equality Deletes with Row Lineage and Define Behavior by @RussellSpitzer in https://github.com/apache/iceberg/pull/12230
• Core: Add "volatile" to HadoopFileIO#hadoopConf by @okumin in https://github.com/apache/iceberg/pull/12388
• Arrow, Parquet, Spark 3.5, Flink 1.20: Avoid deprecated method by @ebyhr in https://github.com/apache/iceberg/pull/11874
• Docs: Add Stackable to the Vendors page by @lfrancke in https://github.com/apache/iceberg/pull/12344
• Build: Upgrade to Gradle 8.13 by @jbonofre in https://github.com/apache/iceberg/pull/12398
• Kafka Connect: Add SMTs for Debezium and AWS DMS by @ismailsimsek in https://github.com/apache/iceberg/pull/11936
• API, Core: Update inclusive metrics evaluator for extract and transforms by @rdblue in https://github.com/apache/iceberg/pull/12311
• Build: remove Hadoop 2 dependency by @Kontinuation in https://github.com/apache/iceberg/pull/12348
• View Spec: Fix typo by @ajantha-bhat in https://github.com/apache/iceberg/pull/12405
• Fix grammar issues in descriptions about Hive environment in hive-quickstart.md by @wenxin-cn in https://github.com/apache/iceberg/pull/12402
• [hive]:Fix Hive table creation syntax errors by @qining-mj in https://github.com/apache/iceberg/pull/12394
• Core: Interface changes for separating rewrite planner and runner by @pvary in https://github.com/apache/iceberg/pull/12306
• Docs: Describe how to handle versioned docs/javadoc during a release by @nastra in https://github.com/apache/iceberg/pull/12413
• Spark: Bump Spark 3.5 to 3.5.5 by @manuzhang in https://github.com/apache/iceberg/pull/12396
• Docs: Remove Hive runtime jar link from latest release by @manuzhang in https://github.com/apache/iceberg/pull/12422
• Docs: Site updates for 1.8.1 by @nastra in https://github.com/apache/iceberg/pull/12410
• Infra: Update Bug report template for 1.8.1 by @nastra in https://github.com/apache/iceberg/pull/12409
• Update release version to 1.8.1 in doap.rdf by @nastra in https://github.com/apache/iceberg/pull/12408
• Build: Ignore docker folder in CI by @manuzhang in https://github.com/apache/iceberg/pull/12417
• Fix IndexOutOfBounds exception in FileFormat#fromFileName by @rshkv in https://github.com/apache/iceberg/pull/12301
• Core: Print un-pretty metadata files without whitespace by @istreeter in https://github.com/apache/iceberg/pull/12318
• Core: Code cleanup around TestTable and TestTableOperations by @gaborkaszab in https://github.com/apache/iceberg/pull/12419
• Data: Add partition stats writer and reader by @ajantha-bhat in https://github.com/apache/iceberg/pull/11216
• Azure: Move docker-based tests to integrationTest by @anuragmantri in https://github.com/apache/iceberg/pull/12274
• spec: fix the wrong example in Geo Types by @jiayuasu in https://github.com/apache/iceberg/pull/12421
• Docs: Fix link of ndv in spark-procedures.md by @wangyinsheng in https://github.com/apache/iceberg/pull/12425
• Build: Bump mkdocs-material from 9.6.5 to 9.6.6 by @dependabot in https://github.com/apache/iceberg/pull/12432
• Build: Ignore README.md/LICENSE/NOTICE in all paths on CI by @manuzhang in https://github.com/apache/iceberg/pull/12429
• Build: Bump org.openapitools:openapi-generator-gradle-plugin from 7.11.0 to 7.12.0 by @dependabot in https://github.com/apache/iceberg/pull/12435
• Build: Bump datamodel-code-generator from 0.28.1 to 0.28.2 by @dependabot in https://github.com/apache/iceberg/pull/12433
• Build: Bump org.mongodb:bson from 4.11.0 to 4.11.5 by @dependabot in https://github.com/apache/iceberg/pull/12438
• Build: Bump slf4j from 2.0.16 to 2.0.17 by @dependabot in https://github.com/apache/iceberg/pull/12436
• Build: Bump jackson-bom from 2.18.2 to 2.18.3 by @dependabot in https://github.com/apache/iceberg/pull/12434
• Fix typo in delete statement: change database name from "dbl" to "db" in spark-writes.md by @wenxin-cn in https://github.com/apache/iceberg/pull/12426
• Build: Bump io.netty:netty-buffer from 4.1.118.Final to 4.1.119.Final by @dependabot in https://github.com/apache/iceberg/pull/12440
• Build: Bump net.snowflake:snowflake-jdbc from 3.22.0 to 3.23.0 by @dependabot in https://github.com/apache/iceberg/pull/12437
• Docs: Deprecate data_file.distinct_counts in v3 by @jacobmarble in https://github.com/apache/iceberg/pull/12182
• Core: Change RemoveSnapshots to remove unused schemas by @gaborkaszab in https://github.com/apache/iceberg/pull/12089
• Core: Add Variant logical type for Avro by @aihuaxu in https://github.com/apache/iceberg/pull/12238
• Spark: Infer partition spec in ADD_FILES procedure for FileTables than taking latest table spec by @bharos in https://github.com/apache/iceberg/pull/12327
• Avro: Support timestamp(9) and unknown types by @rdblue in https://github.com/apache/iceberg/pull/12455
• Wrap variant in PrimitiveHoder so serialization can result same instance by @aihuaxu in https://github.com/apache/iceberg/pull/12317
• Core: Ensure current and newly added view versions are retained in ViewMetadata build by @lliangyu-lin in https://github.com/apache/iceberg/pull/12401
• support create table like in flink catalog by @swapna267 in https://github.com/apache/iceberg/pull/12199
• Parquet: Implement Variant writers by @rdblue in https://github.com/apache/iceberg/pull/12323
• Core: Write null for current-snapshot-id for V3+ by @Fokko in https://github.com/apache/iceberg/pull/12335
• Spec: Add implementation note on current-snapshot-id by @Fokko in https://github.com/apache/iceberg/pull/12334
• Core: Don't create empty RemovePartitionSpecs MetadataUpdate by @gaborkaszab in https://github.com/apache/iceberg/pull/12465
• Data: Expose snapshot-id instead of branch for computing partition stats by @ajantha-bhat in https://github.com/apache/iceberg/pull/12464
• Parquet: Support unknown and timestamp(9) in generics and internal model by @rdblue in https://github.com/apache/iceberg/pull/12463
• Core: Provide access to format-version of metadata table by @nastra in https://github.com/apache/iceberg/pull/12462
• Site: Fix Footer Link by @RussellSpitzer in https://github.com/apache/iceberg/pull/12478
• Docs: fix typo in rest-catalog-open-api.yaml. by @slfan1989 in https://github.com/apache/iceberg/pull/12480
• Build: Bump software.amazon.awssdk:bom from 2.30.26 to 2.30.31 by @dependabot in https://github.com/apache/iceberg/pull/12439
• Build: Bump mkdocs-material from 9.6.6 to 9.6.7 by @dependabot in https://github.com/apache/iceberg/pull/12483
• Build: Bump com.gorylenko.gradle-git-properties:gradle-git-properties from 2.4.2 to 2.5.0 by @dependabot in https://github.com/apache/iceberg/pull/12486
• Build: Rename versions.spark.hive3 to versions.spark3 by @pan3793 in https://github.com/apache/iceberg/pull/12489
• Build: Bump testcontainers from 1.20.5 to 1.20.6 by @dependabot in https://github.com/apache/iceberg/pull/12484
• Core: Apply correct metric configs in GenericAppenderFactory by @XBaith in https://github.com/apache/iceberg/pull/12366
• Add unit test for AddFilesProcedure to check invalid column in partition filter by @bharos in https://github.com/apache/iceberg/pull/12456
• Adds AWS to vendors page by @rbowen in https://github.com/apache/iceberg/pull/12468
• Spark: Rewrite V2 deletes to V3 DVs by @nastra in https://github.com/apache/iceberg/pull/12250
• Docs: Update Iceberg talks with recent Iceberg meetup sessions by @sida-shen in https://github.com/apache/iceberg/pull/12481
• AWS: Integrate S3 analytics accelerator library by @SanjayMarreddi in https://github.com/apache/iceberg/pull/12299
• AWS: Don't fetch credential from endpoint if properties contain a valid credential by @nastra in https://github.com/apache/iceberg/pull/12504
• Revert "OpenAPI: Handle NamespaceNotEmptyException when dropping a namespace" by @nastra in https://github.com/apache/iceberg/pull/12517
• AWS: Update S3 async client configurations and docs for analytics-accelerator-s3 by @SanjayMarreddi in https://github.com/apache/iceberg/pull/12503
• Make reporter() method public so that it can be accessed by Trino for BaseTable creation by @puchengy in https://github.com/apache/iceberg/pull/12519
• Migrate Spark 3.4 test base to JUnit5 by @tomtongue in https://github.com/apache/iceberg/pull/12501
• Core: Fix support for GenericManifestFile index projection by @danielcweeks in https://github.com/apache/iceberg/pull/12522
• Core: Set missing table-default property in RESTSessionCatalog by @ebyhr in https://github.com/apache/iceberg/pull/11646
• Core: Fix default and initial value handling on table creation by @pvary in https://github.com/apache/iceberg/pull/12520
• Core: Don't expose InMemoryViewOperations and RESTViewBuilder outside their visibility scope by @nastra in https://github.com/apache/iceberg/pull/12524
• Kafka Connect: Add config for transactional ID prefix by @thjaeckle in https://github.com/apache/iceberg/pull/11780
• Kafka Connect: Handle no coordinator and data loss in ICR mode by @kumarpritam863 in https://github.com/apache/iceberg/pull/12372
• Core: Use buildKeepingLast for table properties in REST table builder by @smaheshwar-pltr in https://github.com/apache/iceberg/pull/12526
• Flink 1.20: Support Avro and Parquet timestamp(9), unknown, and defaults by @rdblue in https://github.com/apache/iceberg/pull/12470
• Spark 3.4: Backport partition spec inference in spark ADD_FILES procedure by @bharos in https://github.com/apache/iceberg/pull/12508
• Flink 1.18, 1.19: Implement timestamp(9), unknown, and defaults by @rdblue in https://github.com/apache/iceberg/pull/12532
• Build: Bump mkdocs-material from 9.6.7 to 9.6.8 by @dependabot in https://github.com/apache/iceberg/pull/12542
• Build: Bump datamodel-code-generator from 0.28.2 to 0.28.4 by @dependabot in https://github.com/apache/iceberg/pull/12541
• Core: Add missing table-override property to REST catalog by @ebyhr in https://github.com/apache/iceberg/pull/12548
• Spark: Call configureTable in ScanTestBase to ensure proper table configuration by @drexler-sky in https://github.com/apache/iceberg/pull/12546
• AWS, Core, GCP: Auth Manager API enablement by @adutra in https://github.com/apache/iceberg/pull/12197
• Core: fix "Unclosed S3FileIO" warning in JdbcCatalog by @rcjverhoef in https://github.com/apache/iceberg/pull/12540
• Migrate Spark 3.4 TestBase related tests in spark and actions by @tomtongue in https://github.com/apache/iceberg/pull/12552
• API: Implement Variant#toString by @rdblue in https://github.com/apache/iceberg/pull/12531
• Avro: Add variant readers and writers by @rdblue in https://github.com/apache/iceberg/pull/12457
• Core: Add view-override catalog property by @ebyhr in https://github.com/apache/iceberg/pull/12534
• Kafka: Suppress warnings around java.util.Date usage / fix var names by @nastra in https://github.com/apache/iceberg/pull/12561
• REST: HTTPRequest.baseUri() should be nullable by @adutra in https://github.com/apache/iceberg/pull/12556
• Parquet, Core: Enable passing Variant tests by @rdblue in https://github.com/apache/iceberg/pull/12559
• Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled by @hsiang-c in https://github.com/apache/iceberg/pull/12511
• Core, Spark 3.5: Apply Ignore Residuals to Delete Filtering by @RussellSpitzer in https://github.com/apache/iceberg/pull/12479
• Core: Make totalRecordCount optional in PartitionStats by @ajantha-bhat in https://github.com/apache/iceberg/pull/12226
• Core: Replace withFailMessage() with as() by @nastra in https://github.com/apache/iceberg/pull/12570
• Spark: Improve assertions for better debuggability by @nastra in https://github.com/apache/iceberg/pull/12569
• Docs: Update statements mentioning Hive's alpha/beta versions by @okumin in https://github.com/apache/iceberg/pull/12430
• Infra: Update Bug report template for 1.7.2 by @nastra in https://github.com/apache/iceberg/pull/12574
• docs: update go impl status by @zeroshade in https://github.com/apache/iceberg/pull/12578
• Core: Use InternalData when reading manifests in FileCleanupStrategy by @nastra in https://github.com/apache/iceberg/pull/12575
• Docs: Site updates for 1.7.2 by @jbonofre in https://github.com/apache/iceberg/pull/12576
• ORC: Support timestamp(9), variant, and unknown in generics by @rdblue in https://github.com/apache/iceberg/pull/12567
• Flink: Support source watermark for flink sql windows by @swapna267 in https://github.com/apache/iceberg/pull/12191
• Spark: Detect dangling DVs properly by @nastra in https://github.com/apache/iceberg/pull/12270
• OpenAPI: Handle NamespaceNotEmptyException when dropping a namespace by @nastra in https://github.com/apache/iceberg/pull/12518
• Spark: Use correct statistics file in SparkScan::estimateStatistics(Snapshot) by @wypoon in https://github.com/apache/iceberg/pull/12482
• Flink: fix read config of connector.iceberg.max-allowed-planning-failures by @Guosmilesmile in https://github.com/apache/iceberg/pull/12585
• Flink: backport for fix read config of connector.iceberg.max-allowed-planning-failures to 1.18 and 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12589
• Core: Use InternalData with avro for readers. by @danielcweeks in https://github.com/apache/iceberg/pull/12476
• Core, Parquet, ORC: Fix missing data when writing unknown by @rdblue in https://github.com/apache/iceberg/pull/12581
• Core: Handle NamespaceNotEmptyException in NamespaceErrorHandler by @nastra in https://github.com/apache/iceberg/pull/12505
• Spark 3.4: Read DVs when reading from .position_deletes table / Include content offset/size by @nastra in https://github.com/apache/iceberg/pull/12599
• Migrate Spark 3.4 TestBase-related remaining tests in actions by @tomtongue in https://github.com/apache/iceberg/pull/12579
• Spark 3.4: Test metadata tables with format-version=v3 / add ExtensionsTestBase by @nastra in https://github.com/apache/iceberg/pull/12600
• Core: Bulk deletion in RemoveSnapshots by @gaborkaszab in https://github.com/apache/iceberg/pull/11837
• Spark: Backport Spark 3.5 DVs related part to Spark 3.4 by @tomtongue in https://github.com/apache/iceberg/pull/12603
• Parquet: Implement Variant metrics by @rdblue in https://github.com/apache/iceberg/pull/12496
• Spark 3.4: Rewrite data files with high delete ratio by @nastra in https://github.com/apache/iceberg/pull/12601
• Spark 3.4: Rewrite V2 deletes to V3 DVs / Detect dangling DVs properly by @nastra in https://github.com/apache/iceberg/pull/12606
• Build: Bump mkdocs-material from 9.6.8 to 9.6.9 by @dependabot in https://github.com/apache/iceberg/pull/12614
• Build: Bump nessie from 0.103.0 to 0.103.2 by @dependabot in https://github.com/apache/iceberg/pull/12615
• Spec: Geo spec simplifications by @szehon-ho in https://github.com/apache/iceberg/pull/12533
• Spark 3.5: Adjust repeated INFO logs to DEBUG in SparkWrite and SparkPositionDeltaWrite by @manuzhang in https://github.com/apache/iceberg/pull/12404
• Spark 3.4: Backport Spark actions changes in Spark rewrite_table_path procedure (#12006 [#12172] [#11929] [#12282] [#12569]) by @slfan1989 in https://github.com/apache/iceberg/pull/12568
• Core: Add update event for rewrite manifests by @bryanck in https://github.com/apache/iceberg/pull/12627
• Build: Enforce error message check on Exception assertions by @lliangyu-lin in https://github.com/apache/iceberg/pull/12624
• Build: Bump com.google.errorprone:error_prone_annotations from 2.36.0 to 2.37.0 by @dependabot in https://github.com/apache/iceberg/pull/12622
• Core: Add commit metrics for rewriting manifests by @nastra in https://github.com/apache/iceberg/pull/12630
• Data: Refactor PartitionStatsHandler by @ajantha-bhat in https://github.com/apache/iceberg/pull/12550
• Spark 3.4: Propagate snapshot properties / Add max allowed failed commits by @nastra in https://github.com/apache/iceberg/pull/12632
• Docs: Fix lifecycle and versions in multi-engine-support by @manuzhang in https://github.com/apache/iceberg/pull/12370
• Build: Bump parquet from 1.15.0 to 1.15.1 by @dependabot in https://github.com/apache/iceberg/pull/12616
• Build: Bump calcite from 1.10.0 to 1.39.0 by @dependabot in https://github.com/apache/iceberg/pull/12617
• Added New Blog Post: Loading Data into Apache Iceberg by @SourabhEstuary in https://github.com/apache/iceberg/pull/12587
• Core: Add option to fallback to thread classloader by @bryanck in https://github.com/apache/iceberg/pull/12613
• Core: child HTTPClient should not close shared resources by @adutra in https://github.com/apache/iceberg/pull/12566
• API, Core: Add geometry and geography types support by @Kontinuation in https://github.com/apache/iceberg/pull/12346
• AWS: fix incorrect parent session when calling delegate auth manager by @adutra in https://github.com/apache/iceberg/pull/12582
• Docs: Update block spacing guideline in contribute.md by @ajantha-bhat in https://github.com/apache/iceberg/pull/12641
• Docs: Fix ASF sponsorship links by @manuzhang in https://github.com/apache/iceberg/pull/12646
• Spark 3.4: Migrate SparkRowLevelOperationsTestBase related tests to JUnit 5 by @tomtongue in https://github.com/apache/iceberg/pull/12656
• Spark 3.4 : Use correct statistics file in SparkScan::estimateStatistics(Snapshot) by @jeesou in https://github.com/apache/iceberg/pull/12647
• Core: Enhance TestRemoveSnapshots by @manuzhang in https://github.com/apache/iceberg/pull/12662
• update status page for pyiceberg as of 0.9.0 by @kevinjqliu in https://github.com/apache/iceberg/pull/12645
• Build: Bump jetty from 11.0.24 to 11.0.25 by @dependabot in https://github.com/apache/iceberg/pull/12618
• Docs: Fix Latest Iceberg Support version of Hive by @manuzhang in https://github.com/apache/iceberg/pull/12640
• Azure: Support vended credentials refresh in ADLSFileIO. by @ChaladiMohanVamsi in https://github.com/apache/iceberg/pull/11577
• Build: Revert AWS SDK from 2.30.31 to 2.29.52 by @ajantha-bhat in https://github.com/apache/iceberg/pull/12649
• AWS: Use assertThat instead of JUnit4 assertions by @nastra in https://github.com/apache/iceberg/pull/12668
• GCP: Use catalog endpoint as base when refreshing OAuth2 token by @nastra in https://github.com/apache/iceberg/pull/12638
• Core: Cleanup unit tests by @sullis in https://github.com/apache/iceberg/pull/12666
• Core: FileRewritePlanner implementation by @pvary in https://github.com/apache/iceberg/pull/12493
• Flink: backport support create table like in flink catalog by @swapna267 in https://github.com/apache/iceberg/pull/12679
• Doc: Update Instructions for rewrite_table_path. by @slfan1989 in https://github.com/apache/iceberg/pull/12628
• Spark, API: Enhance hashing efficiency by operating on raw UTF-8 bytes by @xiaoxuandev in https://github.com/apache/iceberg/pull/12657
• Build: Bump datamodel-code-generator from 0.28.4 to 0.28.5 by @dependabot in https://github.com/apache/iceberg/pull/12683
• Docs: Update link for User-Defined Tag Restrictions in AWS documentation by @XBaith in https://github.com/apache/iceberg/pull/12698
• Build: Bump com.palantir.gradle.gitversion:gradle-git-version from 3.1.0 to 3.2.0 by @dependabot in https://github.com/apache/iceberg/pull/12687
• Build: Bump com.google.cloud:libraries-bom from 26.55.0 to 26.58.0 by @dependabot in https://github.com/apache/iceberg/pull/12688
• AWS: Fix Catalog URI within VendedCredentialsProvider by @wolflex888 in https://github.com/apache/iceberg/pull/12612
• Core: Add MetricsReporter for SnapshotManager by @wangyinsheng in https://github.com/apache/iceberg/pull/12665
• Spark 3.5: Fix RewriteDataFiles with partial progress enabled and max-failed-commits larger than total-file-group by @manuzhang in https://github.com/apache/iceberg/pull/12120
• Use delimited column names in CreateChangelogViewProcedure by @andyglow in https://github.com/apache/iceberg/pull/12418
• Flink: Backport support source watermark for flink sql windows by @swapna267 in https://github.com/apache/iceberg/pull/12697
• Spark 3.4: Fix RewriteDataFiles with partial progress enabled and max-failed-commits larger than total-file-group by @manuzhang in https://github.com/apache/iceberg/pull/12701
• Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4.2 to 5.4.3 by @dependabot in https://github.com/apache/iceberg/pull/12685
• Build: Bump guava from 33.4.0-jre to 33.4.6-jre by @dependabot in https://github.com/apache/iceberg/pull/12686
• Core: Pass storage credentials from LoadTableResponse to FileIO by @nastra in https://github.com/apache/iceberg/pull/12591
• AWS: Update the aws-bundle with latest dependencies by @SanjayMarreddi in https://github.com/apache/iceberg/pull/12553
• Core, Hive: Double check commit status in case of commit conflict for NoLock by @lirui-apache in https://github.com/apache/iceberg/pull/12637
• Spark 3.4: Fix NotSerializableException when migrating Spark tables by @manuzhang in https://github.com/apache/iceberg/pull/12705
• Doc: Remove Warning for resolved issue [#11147]. by @slfan1989 in https://github.com/apache/iceberg/pull/12694
• Flink: Backport avoid RANGE mode broken chain when write parallelism changes by @huyuanfeng2018 in https://github.com/apache/iceberg/pull/12080
• Updated vendors documentation to add SingleStore by @akoller in https://github.com/apache/iceberg/pull/12708
• Spec: update to reflect lineage is required by @danielcweeks in https://github.com/apache/iceberg/pull/12580
• Core: Enhance remove snapshots efficiency by executing them in bulk by @ricardopereira33 in https://github.com/apache/iceberg/pull/12670
• Core: Update deprecation msg by @nastra in https://github.com/apache/iceberg/pull/12720
• AWS: Add AWS integ tests to check task and enable tests based on required environment variables by @lliangyu-lin in https://github.com/apache/iceberg/pull/12671
• Core, Spark: Add row lineage metadata columns, and surface them in SparkTable metadata columns by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12596
• Build: Bump mkdocs-material from 9.6.9 to 9.6.11 by @dependabot in https://github.com/apache/iceberg/pull/12728
• Build: Bump io.delta:delta-standalone_2.12 from 3.3.0 to 3.3.1 by @dependabot in https://github.com/apache/iceberg/pull/12731
• Build: Bump net.snowflake:snowflake-jdbc from 3.23.0 to 3.23.2 by @dependabot in https://github.com/apache/iceberg/pull/12732
• Flink: Backport RowConverter to Flink 1.19 and 1.18 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12713
• Flink: Fix npe in SketchUtil when numPartitions bigger than length of samples by @Guosmilesmile in https://github.com/apache/iceberg/pull/12703
• Core: Lazy init workerPool in RemoveSnapshots and SnapshotProducer by @abstractdog in https://github.com/apache/iceberg/pull/12427
• Flink: Backport Typeinfo in RowConverter to Flink 1.19 and 1.18 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12739
• Hive: Refactor HMS table parameter setting to be able to reuse by @zratkai in https://github.com/apache/iceberg/pull/12461
• Flink:Backport fix npe in SketchUtil to Flink 1.19 and 1.18 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12741
• Core: Enable row lineage for all v3 tables by @rdblue in https://github.com/apache/iceberg/pull/12593
• Migrate Spark 3.4 ExtensionsTestBase-related tests by @tomtongue in https://github.com/apache/iceberg/pull/12744
• Build: Bump io.delta:delta-spark_2.12 from 3.3.0 to 3.3.1 by @dependabot in https://github.com/apache/iceberg/pull/12729
• Doc: Remove Hive 2.x/3.x related docs in hive.md by @jackylee-ch in https://github.com/apache/iceberg/pull/12700
• Core: Return this instead of null in enableRowLineage() by @nastra in https://github.com/apache/iceberg/pull/12747
• Core: Drop invalid function comment for HTTPClient.isSuccessful by @gaborkaszab in https://github.com/apache/iceberg/pull/12742
• CORE: Allow HTTPClient to parse headers from properties. by @wolflex888 in https://github.com/apache/iceberg/pull/12595
• Throw on {write.folder-storage.path,write.object-storage.path} properties by @Fokko in https://github.com/apache/iceberg/pull/12315
• Build: Bump io.netty:netty-buffer from 4.1.119.Final to 4.2.0.Final by @dependabot in https://github.com/apache/iceberg/pull/12730
• API: Deprecate CredentialSupplier by @nastra in https://github.com/apache/iceberg/pull/12763
• AWS: Add unit tests for AWS s3Async by @sullis in https://github.com/apache/iceberg/pull/12758
• Build: Bump com.google.cloud:libraries-bom from 26.58.0 to 26.59.0 by @dependabot in https://github.com/apache/iceberg/pull/12733
• Upgrades Flink Version to 1.19.2 and 1.20.1 by @rodmeneses in https://github.com/apache/iceberg/pull/12745
• Update hive-quickstart.md by @mollonado in https://github.com/apache/iceberg/pull/12746
• Flink: fix rateLimit argument check in TableMaintenance by @Guosmilesmile in https://github.com/apache/iceberg/pull/12773
• Flink: backport fix rateLimit argument check in TableMaintenance to Flink 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12776
• Docs: Add Estuary to docs and vendors by @aeluce in https://github.com/apache/iceberg/pull/12764
• Core: Update RewriteFiles tests to test against V3 by @amogh-jahagirdar in https://github.com/apache/iceberg/pull/12777
• Migrate Spark 3.4 ExtensionsTestBase-related tests for Partition, Schema and Branch/Tag by @tomtongue in https://github.com/apache/iceberg/pull/12766
• Build: Bump guava from 33.4.6-jre to 33.4.7-jre by @dependabot in https://github.com/apache/iceberg/pull/12789
• Build: Bump nessie from 0.103.2 to 0.103.3 by @dependabot in https://github.com/apache/iceberg/pull/12786
• Core: Use OutputFile.location(), InputFile.location() in Error Messages by @Jordano-Dremio in https://github.com/apache/iceberg/pull/12755
• Flink: Move unlock from MemoryLock open to TestCase Before by @Guosmilesmile in https://github.com/apache/iceberg/pull/12793
• Flink: backport [#12793] to Flink 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12795
• Build: Bump junit to 5.12.2 by @iProdigy in https://github.com/apache/iceberg/pull/12391
• Flink: Fix TriggerManager to unlock task execution when previous job left an orphaned lock by @Guosmilesmile in https://github.com/apache/iceberg/pull/12794
• Flink: backport fix TriggerManager to unlock task execution when previous job left an orphaned lock for Flink 1.19 by @Guosmilesmile in https://github.com/apache/iceberg/pull/12801
• Flink: fix parameters for test recovery by @Guosmilesmile in https://github.com/apache/iceberg/pull/12800
• Spark 3.4: Migrate integration test to JUnit5 by @nastra in https://github.com/apache/iceberg/pull/12796
• CatalogTests: Fix listNamespaces Check, Avoid Reserved Keyword, Allow Configurable Location by @talatuyarer in https://github.com/apache/iceberg/pull/12768
• Update the docs for working with Flink and REST catalog by @dttung2905 in https://github.com/apache/iceberg/pull/12726
• Migrate Spark 3.4 ExtensionsTestBase-related tests for Snapshot manipulation, ChangeLogView and Distribution/Ordering by @tomtongue in https://github.com/apache/iceberg/pull/12807
• Core: Fix deprecated FileSystem.isDirectory warning and remove redundant test code by @ebyhr in https://github.com/apache/iceberg/pull/12805
• Core: Test loading table/view with non-existing namespace by @nastra in https://github.com/apache/iceberg/pull/12812
• Doc: Fix typo in spark-procedures.md by @slfan1989 in https://github.com/apache/iceberg/pull/12814
• Spark 3.4: Migrate ExtensionsTestBase-related remaining tests by @tomtongue in https://github.com/apache/iceberg/pull/12813
• Docs: Add the recommended style for ArrayAssertions by @tomtongue in https://github.com/apache/iceberg/pull/12820
• Spark 3.5: Use ProcedureInput for SnapshotTableProcedure. by @slfan1989 in https://github.com/apache/iceberg/pull/12783
• Spark 3.5: Use ProcedureInput for MigrateTableProcedure. by @slfan1989 in https://github.com/apache/iceberg/pull/12782
• Fix versions in LICENSE and NOTICE by @jbonofre in https://github.com/apache/iceberg/pull/12831
• Spark3.4: Backport ProcedureInput for MigrateTableProcedure And SnapshotTableProcedure (#12782 [#12783]) by @slfan1989 in https://github.com/apache/iceberg/pull/12837
• Core: Support first-row-id for manifests and manifest lists by @rdblue in https://github.com/apache/iceberg/pull/12672
• Site: Remove Iceberg Summit Link from the Homepage by @RussellSpitzer in https://github.com/apache/iceberg/pull/12842
• Core: use ALL_VERSIONS constant in TestBase by @sullis in https://github.com/apache/iceberg/pull/12748
• Spec: Update row lineage requirements for upgrading tables by @rdblue in https://github.com/apache/iceberg/pull/12781
• Flink: Add StreamingStartingStrategy.INCREMENTAL_FROM_LATEST_SNAPSHOT_EXCLUSIVE by @morhidi in https://github.com/apache/iceberg/pull/12839
• Spark: Add _row_id and _last_updated_sequence_number readers by @rdblue in https://github.com/apache/iceberg/pull/12836
• spec: Variant lower/upper bounds by @aihuaxu in https://github.com/apache/iceberg/pull/12658
• API: Use normalized JSON path to identify Variant fields by @rdblue in https://github.com/apache/iceberg/pull/12835
• Core: Add test cases for row lineage metadata by @rdblue in https://github.com/apache/iceberg/pull/12843
• Spark 3.5: Add Parallelism Parameter Validation to AddFilesProcedure. by @slfan1989 in https://github.com/apache/iceberg/pull/12784
• Spec: Allow the use of source-id in V3 by @Fokko in https://github.com/apache/iceberg/pull/12644
• Core: Use assumeThat instead of assumeTrue by @slfan1989 in https://github.com/apache/iceberg/pull/12822
• Spark3.4: Migrate tests in spark, extensions and functions by @tomtongue in https://github.com/apache/iceberg/pull/12853
• API: Don't check underlying error msg on AIOOBE by @nastra in https://github.com/apache/iceberg/pull/12867

New Contributors

• @danicafine made their first contribution in https://github.com/apache/iceberg/pull/12256
• @czy006 made their first contribution in https://github.com/apache/iceberg/pull/11966
• @lfrancke made their first contribution in https://github.com/apache/iceberg/pull/12344
• @wenxin-cn made their first contribution in https://github.com/apache/iceberg/pull/12402
• @qining-mj made their first contribution in https://github.com/apache/iceberg/pull/12394
• @istreeter made their first contribution in https://github.com/apache/iceberg/pull/12318
• @jiayuasu made their first contribution in https://github.com/apache/iceberg/pull/12421
• @bharos made their first contribution in https://github.com/apache/iceberg/pull/12327
• @swapna267 made their first contribution in https://github.com/apache/iceberg/pull/12199
• @XBaith made their first contribution in https://github.com/apache/iceberg/pull/12366
• @rbowen made their first contribution in https://github.com/apache/iceberg/pull/12468
• @sida-shen made their first contribution in https://github.com/apache/iceberg/pull/12481
• @SanjayMarreddi made their first contribution in https://github.com/apache/iceberg/pull/12299
• @thjaeckle made their first contribution in https://github.com/apache/iceberg/pull/11780
• @kumarpritam863 made their first contribution in https://github.com/apache/iceberg/pull/12372
• @drexler-sky made their first contribution in https://github.com/apache/iceberg/pull/12546
• @SourabhEstuary made their first contribution in https://github.com/apache/iceberg/pull/12587
• @ChaladiMohanVamsi made their first contribution in https://github.com/apache/iceberg/pull/11577
• @wolflex888 made their first contribution in https://github.com/apache/iceberg/pull/12612
• @andyglow made their first contribution in https://github.com/apache/iceberg/pull/12418
• @akoller made their first contribution in https://github.com/apache/iceberg/pull/12708
• @ricardopereira33 made their first contribution in https://github.com/apache/iceberg/pull/12670
• @abstractdog made their first contribution in https://github.com/apache/iceberg/pull/12427
• @zratkai made their first contribution in https://github.com/apache/iceberg/pull/12461
• @jackylee-ch made their first contribution in https://github.com/apache/iceberg/pull/12700
• @mollonado made their first contribution in https://github.com/apache/iceberg/pull/12746
• @aeluce made their first contribution in https://github.com/apache/iceberg/pull/12764
• @Jordano-Dremio made their first contribution in https://github.com/apache/iceberg/pull/12755
• @iProdigy made their first contribution in https://github.com/apache/iceberg/pull/12391
• @talatuyarer made their first contribution in https://github.com/apache/iceberg/pull/12768
• @dttung2905 made their first contribution in https://github.com/apache/iceberg/pull/12726
• @morhidi made their first contribution in https://github.com/apache/iceberg/pull/12839

Full Changelog: https://github.com/apache/iceberg/compare/apache-iceberg-1.8.0...apache-iceberg-1.9.0