Pass headers down instead of using a block

[tenderlove]

Right now we are buffering entire gems in memory when we download them.

• https://github.com/rubygems/rubygems/blob/66c94f65c8a05484f2d12cb0bccd8513da5c65ef/lib/rubygems/request.rb#L208-L216

I would like to open the possibility of streaming assets rather than storing the entire asset in memory. In order to accomplish this, I would like to allow callers to pass a block to indicate that they "want to stream".

Before this commit, we would use the block to set headers on the request object. This commit passes the headers down to construct the request object. Callers know the headers they want to set before calling the method, so it makes sense to pass them in via parameters rather than as a block

[Edouard-chin]

I think we'll need to modify this one too

rubygems/bundler/lib/bundler/fetcher/gem_remote_fetcher.rb

Lines 8 to 11 in 48272fb

	def request(*args)
	super do |req|
	req.delete("User-Agent") if headers["User-Agent"]
	yield req if block_given?

[colby-swandale]

I believe this method will also need updating

rubygems/lib/rubygems/gemcutter_utilities.rb

Lines 249 to 258 in da4a289

	def request_with_otp(method, uri, &block)
	request_method = Gem::Net::HTTP.const_get method.to_s.capitalize
	Gem::RemoteFetcher.fetcher.request(uri, request_method) do |req|
	req["OTP"] = otp if otp
	block.call(req)
	end
	ensure
	options[:otp] = nil if webauthn_enabled?
	end

[hsbt]

@tenderlove Can you look #9021 (comment)?