Skip to content

Commit 35aefaf

Browse files
authored
fix(glue-alpha): correct OutputFormat.AVRO/ORC type and tidy GA-hygiene items (#38593)
`OutputFormat.AVRO` and `OutputFormat.ORC` were constructed as `InputFormat` instances — they compiled only because the two classes are structurally identical. They are now `OutputFormat` instances; the emitted Hadoop class-name strings are unchanged, so synthesized output is identical. Also: - `renderColumns` typed as the malformed `Array<Column | Column>` -> `Column[]` in both `S3Table` and `ExternalTable`. - `Database.locationUri` is now `readonly` (it is assigned only at construction). BREAKING CHANGE: this is a corrective breaking change. Apps that leaned on the bug, and did things like `InputFormat x = OutputFormat.AVRO;` will get a compilation error in other jsii languages. The intended usage, on the other hand, was broken before and works now. ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent 5f3b1b6 commit 35aefaf

5 files changed

Lines changed: 25 additions & 5 deletions

File tree

‎packages/@aws-cdk/aws-glue-alpha/lib/data-format.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,14 +59,14 @@ export class OutputFormat {
5959
*
6060
* @see https://svn.apache.org/repos/infra/websites/production/hive/content/javadocs/r3.1.3/api/org/apache/hadoop/hive/ql/io/avro/AvroContainerOutputFormat.html
6161
*/
62-
public static readonly AVRO = new InputFormat('org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat');
62+
public static readonly AVRO = new OutputFormat('org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat');
6363

6464
/**
6565
* OutputFormat for Orc files.
6666
*
6767
* @see https://svn.apache.org/repos/infra/websites/production/hive/content/javadocs/r3.1.3/api/org/apache/hadoop/hive/ql/io/orc/OrcOutputFormat.html
6868
*/
69-
public static readonly ORC = new InputFormat('org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat');
69+
public static readonly ORC = new OutputFormat('org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat');
7070

7171
/**
7272
* OutputFormat for Parquet files.

‎packages/@aws-cdk/aws-glue-alpha/lib/database.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,7 @@ export class Database extends Resource implements IDatabase {
100100
/**
101101
* Location URI of this database.
102102
*/
103-
public locationUri?: string;
103+
public readonly locationUri?: string;
104104

105105
private readonly _catalog?: ICatalog;
106106

‎packages/@aws-cdk/aws-glue-alpha/lib/external-table.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ const writePermissions = [
185185
'glue:UpdatePartition',
186186
];
187187

188-
function renderColumns(columns?: Array<Column | Column>) {
188+
function renderColumns(columns?: Column[]) {
189189
if (columns === undefined) {
190190
return undefined;
191191
}

‎packages/@aws-cdk/aws-glue-alpha/lib/s3-table.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -345,7 +345,7 @@ function createBucket(table: S3Table, props: S3TableProps) {
345345
};
346346
}
347347

348-
function renderColumns(columns?: Array<Column | Column>) {
348+
function renderColumns(columns?: Column[]) {
349349
if (columns === undefined) {
350350
return undefined;
351351
}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import { InputFormat, OutputFormat } from '../lib';
2+
3+
// Regression test: OutputFormat.AVRO and OutputFormat.ORC were previously
4+
// constructed as `InputFormat` instances. That compiled in TypeScript because
5+
// the two classes are structurally identical, but in jsii's nominally-typed
6+
// languages (Java, C#, Go, Python) it made these members unusable wherever an
7+
// `OutputFormat` is required. At runtime the classes have distinct prototypes,
8+
// so `instanceof` distinguishes the bug (would fail below) from the fix.
9+
describe('OutputFormat static members are OutputFormat instances', () => {
10+
test.each([
11+
['AVRO', OutputFormat.AVRO, 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'],
12+
['ORC', OutputFormat.ORC, 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'],
13+
['HIVE_IGNORE_KEY_TEXT', OutputFormat.HIVE_IGNORE_KEY_TEXT, 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'],
14+
['PARQUET', OutputFormat.PARQUET, 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'],
15+
])('OutputFormat.%s is an OutputFormat with an unchanged class name', (_name, format, className) => {
16+
expect(format).toBeInstanceOf(OutputFormat);
17+
expect(format).not.toBeInstanceOf(InputFormat);
18+
expect(format.className).toEqual(className);
19+
});
20+
});

0 commit comments

Comments
 (0)