72 commits

Author	SHA1	Message	Date
Michael Yang	630bb75d2a	dynamically size download parts based on file size	2023-10-11 14:10:25 -07:00
Michael Yang	a2055a1e93	update download	2023-10-11 14:10:25 -07:00
Jeffrey Morgan	65dcd0ce35	always cleanup blob download (#747)	2023-10-10 13:12:29 -04:00
Michael Yang	0560b28a8d	names	2023-10-06 12:56:56 -07:00
Michael Yang	10199c5987	replace done channel with file check	2023-10-06 12:56:56 -07:00
Michael Yang	288814d3e4	fix ref counts	2023-10-06 12:56:43 -07:00
Michael Yang	04733438da	check head request response	2023-10-06 12:56:43 -07:00
Michael Yang	711e891f0f	fix resumable downloads ... glob returns files in lexical order which is not appropriate when rebuilding the parts list	2023-10-06 12:56:43 -07:00
Michael Yang	090d08422b	handle unexpected eofs	2023-10-06 12:56:43 -07:00
Michael Yang	5b84404c64	handle concurrent requests for the same blobs	2023-10-06 12:56:43 -07:00
Michael Yang	8544edca21	parallel chunked downloads	2023-10-06 12:56:43 -07:00
Michael Yang	f40b3de758	use int64 consistently	2023-09-28 11:07:24 -07:00
Michael Yang	b58d5d16b0	fix mkdir on windows	2023-09-19 09:41:13 -07:00
Michael Yang	246dc65417	loosen http status code checks	2023-08-28 18:34:53 -04:00
Michael Yang	2cc634689b	use url.URL	2023-08-22 10:49:07 -07:00
Michael Yang	2c7f956b38	add version	2023-08-22 09:40:58 -07:00
Bruce MacDonald	f0d7c2f5ea	retry download on network errors	2023-08-15 15:07:19 -03:00
Bruce MacDonald	f020e1d519	always remove from in progress map on download	2023-08-14 13:09:20 -03:00
Patrick Devine	d9cf18e28d	add maximum retries when pushing (#334)	2023-08-11 15:41:55 -07:00
Bruce MacDonald	8228d166ce	pr comments	2023-08-09 11:31:54 -04:00
Bruce MacDonald	907e6c56b3	unlock downloadu in case or requestDownload err	2023-08-09 11:31:54 -04:00
Bruce MacDonald	868e3b31c7	allow for concurrent pulls of the same files	2023-08-09 11:31:54 -04:00