Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
042dca2
fix(storage): standardize URL formatting and enhance transport retry
thiyaguk09 May 7, 2026
5f18f7e
fix(storage): resolve transport and retry issues (#8235)
thiyaguk09 Jun 23, 2026
288e31a
lint fix
thiyaguk09 Jun 23, 2026
fbedc7c
fix(storage): Invocation ID is not retained on multipart upload retri…
thiyaguk09 Jul 28, 2026
053428f
test: update resumable upload test mocks to use URL and Headers objects
thiyaguk09 Aug 27, 2026
9764d21
style: apply prettier formatting throughout the codebase to ensure co…
thiyaguk09 Aug 28, 2026
b20961b
refactor: improve type safety and remove any casts across storage tra…
thiyaguk09 Aug 28, 2026
d8b93e8
refactor: move upload initialization into the writing event pipeline …
thiyaguk09 Aug 28, 2026
8576398
test(storage): system tests for gaxios migration
thiyaguk09 Aug 28, 2026
4a73eb4
fix(storage): standardize URL formatting and enhance transport retry
thiyaguk09 May 7, 2026
320b5cc
fix(storage): resolve transport and retry issues (#8235)
thiyaguk09 Jun 23, 2026
c2710bf
lint fix
thiyaguk09 Jun 23, 2026
463e7f8
fix(storage): Invocation ID is not retained on multipart upload retri…
thiyaguk09 Jul 28, 2026
ca30b2e
test: update resumable upload test mocks to use URL and Headers objects
thiyaguk09 Aug 27, 2026
01e1f17
style: apply prettier formatting throughout the codebase to ensure co…
thiyaguk09 Aug 28, 2026
d104ded
refactor: improve type safety and remove any casts across storage tra…
thiyaguk09 Aug 28, 2026
b541179
refactor: move upload initialization into the writing event pipeline …
thiyaguk09 Aug 28, 2026
2e5eee3
Merge branch 'storage-gaxios-migration' into test/storage-system-gaxios
thiyaguk09 Aug 31, 2026
87e8b01
fix(storage): standardize URL formatting and enhance transport retry
thiyaguk09 May 7, 2026
79f93ab
fix(storage): resolve transport and retry issues (#8235)
thiyaguk09 Jun 23, 2026
620e0c0
lint fix
thiyaguk09 Jun 23, 2026
2476994
fix(storage): Invocation ID is not retained on multipart upload retri…
thiyaguk09 Jul 28, 2026
25de310
test: update resumable upload test mocks to use URL and Headers objects
thiyaguk09 Aug 27, 2026
854011f
style: apply prettier formatting throughout the codebase to ensure co…
thiyaguk09 Aug 28, 2026
117ea85
refactor: improve type safety and remove any casts across storage tra…
thiyaguk09 Aug 28, 2026
9b97153
refactor: move upload initialization into the writing event pipeline …
thiyaguk09 Aug 28, 2026
7823f87
chore: update test formatting and refactor IP filter metadata tests t…
thiyaguk09 Aug 31, 2026
3ead0d8
refactor: update header type definitions in resumable upload tests to…
thiyaguk09 Aug 31, 2026
74a48d8
Merge remote-tracking branch 'upstream/storage-gaxios-migration' into…
thiyaguk09 Sep 1, 2026
1b2f4d3
fix: improve header handling and resolve minor metadata and request bugs
thiyaguk09 Sep 1, 2026
aa90879
fix(storage): standardize URL formatting and enhance transport retry
thiyaguk09 May 7, 2026
589ee20
fix(storage): resolve transport and retry issues (#8235)
thiyaguk09 Jun 23, 2026
b7a71e7
lint fix
thiyaguk09 Jun 23, 2026
3a9c6f6
fix(storage): Invocation ID is not retained on multipart upload retri…
thiyaguk09 Jul 28, 2026
11d30cb
test: update resumable upload test mocks to use URL and Headers objects
thiyaguk09 Aug 27, 2026
aa72b03
style: apply prettier formatting throughout the codebase to ensure co…
thiyaguk09 Aug 28, 2026
5c7fb31
refactor: improve type safety and remove any casts across storage tra…
thiyaguk09 Aug 28, 2026
8d3f0f0
refactor: move upload initialization into the writing event pipeline …
thiyaguk09 Aug 28, 2026
bf72fb2
chore: update test formatting and refactor IP filter metadata tests t…
thiyaguk09 Aug 31, 2026
3eb2445
refactor: update header type definitions in resumable upload tests to…
thiyaguk09 Aug 31, 2026
658c481
Merge branch 'storage-gaxios-migration' into test/storage-system-gaxios
thiyaguk09 Sep 3, 2026
85958f4
fix(storage): standardize URL formatting and enhance transport retry
thiyaguk09 May 7, 2026
251a44b
fix(storage): resolve transport and retry issues (#8235)
thiyaguk09 Jun 23, 2026
9f4e5ea
lint fix
thiyaguk09 Jun 23, 2026
e41e310
fix(storage): Invocation ID is not retained on multipart upload retri…
thiyaguk09 Jul 28, 2026
93dc2fe
test: update resumable upload test mocks to use URL and Headers objects
thiyaguk09 Aug 27, 2026
da25dc9
style: apply prettier formatting throughout the codebase to ensure co…
thiyaguk09 Aug 28, 2026
f99a09d
refactor: improve type safety and remove any casts across storage tra…
thiyaguk09 Aug 28, 2026
74cc740
refactor: move upload initialization into the writing event pipeline …
thiyaguk09 Aug 28, 2026
b03ef5a
chore: update test formatting and refactor IP filter metadata tests t…
thiyaguk09 Aug 31, 2026
d749fc2
refactor: update header type definitions in resumable upload tests to…
thiyaguk09 Aug 31, 2026
a9d039b
Merge branch 'storage-gaxios-migration' into test/storage-system-gaxios
thiyaguk09 Sep 4, 2026
cbb389c
refactor: update header type definitions in resumable upload tests to…
thiyaguk09 Aug 31, 2026
f222aaf
Merge branch 'storage-gaxios-migration' into test/storage-system-gaxios
thiyaguk09 Sep 7, 2026
2072d69
refactor: update header type definitions in resumable upload tests to…
thiyaguk09 Aug 31, 2026
b9e03d3
Merge branch 'storage-gaxios-migration' into test/storage-system-gaxios
thiyaguk09 Sep 7, 2026
bef59ce
test: update IAM policy tests and refine error validation logic in st…
thiyaguk09 Sep 7, 2026
ec8d817
Merge branch 'storage-gaxios-migration' into test/storage-system-gaxios
thiyaguk09 Sep 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion handwritten/storage/src/bucket.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1603,7 +1603,7 @@ class Bucket extends ServiceObject<Bucket, BucketMetadata> {
return;
}

const currentLifecycleRules = Array.isArray(metadata.lifecycle?.rule)
const currentLifecycleRules = Array.isArray(metadata?.lifecycle?.rule)
? metadata.lifecycle?.rule
: [];

Expand Down
14 changes: 11 additions & 3 deletions handwritten/storage/src/nodejs-common/service-object.ts
Original file line number Diff line number Diff line change
Expand Up @@ -492,7 +492,9 @@ class ServiceObject<T, K extends BaseMetadata> extends EventEmitter {
},
},
(err, data, resp) => {
this.metadata = data!;
if (!err && data) {
this.metadata = data;
}
callback(err, data!, resp);
},
)
Expand Down Expand Up @@ -558,8 +560,14 @@ class ServiceObject<T, K extends BaseMetadata> extends EventEmitter {
},
},
(err, data, resp) => {
this.metadata = data!;
callback(err, this.metadata, resp);
if (!err && data) {
this.metadata = data;
}
callback(
err,
(err ? undefined : this.metadata) as unknown as K,
resp,
);
},
)
// eslint-disable-next-line promise/no-callback-in-promise
Expand Down
16 changes: 15 additions & 1 deletion handwritten/storage/src/nodejs-common/util.ts
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,21 @@ export function decorateHeaders(
headers?: Headers,
options?: DecorateHeadersOptions,
): DecorateHeadersResult {
const sanitizedHeaders: Headers = {...headers};
const sanitizedHeaders: Headers = {};
if (headers) {
if (
typeof (headers as Headers & {entries?: () => Iterable<[string, string]>})
.entries === 'function'
) {
for (const [key, value] of (
headers as Headers & {entries: () => Iterable<[string, string]>}
).entries()) {
sanitizedHeaders[key] = value;
}
} else {
Object.assign(sanitizedHeaders, headers);
}
}
const userTokenKey = Object.keys(sanitizedHeaders).find(
key => key.toLowerCase() === 'x-goog-gcs-idempotency-token',
);
Expand Down
31 changes: 25 additions & 6 deletions handwritten/storage/src/storage-transport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,20 @@ export class StorageTransport {
hasEtagInBody
);

// Helper to enrich GaxiosError objects with legacy ApiError properties
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const decorateError = (err: any) => {
if (err && typeof err === 'object') {
err.code = err.response?.status || err.status || err.code;
if (err.response?.data?.error) {
const apiError = err.response.data.error;
if (apiError.message) err.message = apiError.message;
if (apiError.errors) err.errors = apiError.errors;
}
}
return err;
};

try {
const requestPromise = this.authClient.request<T>({
adapter: async (opts: GaxiosOptions) => {
Expand Down Expand Up @@ -235,11 +249,15 @@ export class StorageTransport {
return data;
};

const enrichedPromise = requestPromise.catch(err => {
throw decorateError(err);
});

if (callback) {
// eslint-disable-next-line @typescript-eslint/no-floating-promises
(async () => {
try {
const resp = await requestPromise;
const resp = await enrichedPromise;
callback(null, decorateMetadata(resp), resp);
} catch (err: unknown) {
callback(
Expand All @@ -249,16 +267,17 @@ export class StorageTransport {
);
}
})();
return requestPromise;
return enrichedPromise;
}

return requestPromise;
return enrichedPromise;
} catch (e) {
const err = decorateError(e);
if (callback) {
callback(e as GaxiosError);
return Promise.reject(e);
callback(err as GaxiosError);
return Promise.reject(err);
}
throw e;
throw err;
}
}

Expand Down
8 changes: 4 additions & 4 deletions handwritten/storage/src/storage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1205,15 +1205,15 @@ export class Storage {
'Content-Type': 'application/json',
},
},
(err, data, resp) => {
(err, data) => {
if (err) {
callback(err);
return;
}
const bucket = this.bucket(name);
bucket.metadata = data!;

callback(null, bucket, resp);
callback(null, bucket, data);
},
)
.catch(err => callback!(err));
Expand Down Expand Up @@ -1331,7 +1331,7 @@ export class Storage {
retry: false,
responseType: 'json',
},
(err, data, resp) => {
(err, data) => {
if (err) {
callback(err);
return;
Expand All @@ -1347,7 +1347,7 @@ export class Storage {
null,
hmacKey,
hmacKey.secret,
resp as unknown as HmacKeyResourceResponse,
data as HmacKeyResourceResponse,
);
},
)
Expand Down
3 changes: 1 addition & 2 deletions handwritten/storage/system-test/fixtures/index-cjs.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,10 @@
// See the License for the specific language governing permissions and
// limitations under the License.

// eslint-disable-next-line no-undef
/* eslint-disable node/no-missing-require, no-unused-vars, no-undef */
const {Storage} = require('@google-cloud/storage');

function main() {
// eslint-disable-next-line no-unused-vars
const storage = new Storage();
}

Expand Down
5 changes: 2 additions & 3 deletions handwritten/storage/system-test/fixtures/index-esm.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,10 @@
// See the License for the specific language governing permissions and
// limitations under the License.

// eslint-disable-next-line no-undef
const {Storage} = require('@google-cloud/storage');
/* eslint-disable node/no-missing-import, no-unused-vars */
import {Storage} from '@google-cloud/storage';

function main() {
// eslint-disable-next-line no-unused-vars
const storage = new Storage();
}

Expand Down
2 changes: 1 addition & 1 deletion handwritten/storage/system-test/install.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ describe('pack-n-play tests', () => {
await packNTest({
sample: {
description: 'Should be able to import the storage library in ESM',
cjs: readFileSync('./system-test/fixtures/index-esm.js').toString(),
esm: readFileSync('./system-test/fixtures/index-esm.js').toString(),
},
});
});
Expand Down
7 changes: 5 additions & 2 deletions handwritten/storage/system-test/kitchen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,10 @@ describe('resumable-upload', () => {
retryableErrorFn: RETRYABLE_ERR_FN_DEFAULT,
};

const bucket = new Storage({retryOptions}).bucket(bucketName);
const bucket = new Storage({
projectId: process.env.PROJECT_ID,
retryOptions: retryOptions,
}).bucket(bucketName);
let filePath: string;

before(async () => {
Expand Down Expand Up @@ -97,7 +100,7 @@ describe('resumable-upload', () => {
// see: https://cloud.google.com/storage/docs/exponential-backoff:
const ms = Math.pow(2, retries) * 1000 + Math.random() * 2000;
console.info(`retrying "${title}" in ${ms}ms`);
setTimeout(done(), ms);
setTimeout(() => { done(); }, ms);
}

it('should work', done => {
Expand Down
Loading
Loading