Module: Homebrew::Livecheck::Strategy Private

Extended by:
Utils::Curl
Defined in:
livecheck/strategy.rb,
livecheck/strategy/git.rb,
livecheck/strategy/gnu.rb,
livecheck/strategy/npm.rb,
livecheck/strategy/xml.rb,
livecheck/strategy/cpan.rb,
livecheck/strategy/json.rb,
livecheck/strategy/pypi.rb,
livecheck/strategy/xorg.rb,
livecheck/strategy/yaml.rb,
livecheck/strategy/crate.rb,
livecheck/strategy/gnome.rb,
livecheck/strategy/apache.rb,
livecheck/strategy/hackage.rb,
livecheck/strategy/sparkle.rb,
livecheck/strategy/bitbucket.rb,
livecheck/strategy/launchpad.rb,
livecheck/strategy/page_match.rb,
livecheck/strategy/sourceforge.rb,
livecheck/strategy/header_match.rb,
livecheck/strategy/extract_plist.rb,
livecheck/strategy/github_latest.rb,
livecheck/strategy/github_releases.rb,
livecheck/strategy/electron_builder.rb

Overview

This module is part of a private API. This module may only be used in the Homebrew/brew repository. Third parties should avoid using this module if possible, as it may be removed or changed without warning.

The Livecheck::Strategy module contains the various strategies as well as some general-purpose methods for working with them. Within the context of the brew livecheck command, strategies are established procedures for finding new software versions at a given source.

Defined Under Namespace

Classes: Apache, Bitbucket, Cpan, Crate, ElectronBuilder, ExtractPlist, Git, GithubLatest, GithubReleases, Gnome, Gnu, Hackage, HeaderMatch, Json, Launchpad, Npm, PageMatch, Pypi, Sourceforge, Sparkle, Xml, Xorg, Yaml

Constant Summary collapse

DEFAULT_PRIORITY =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

Homebrew::Livecheck::Strategy priorities informally range from 1 to 10, where 10 is the highest priority. 5 is the default priority because it's roughly in the middle of this range. Strategies with a priority of 0 (or lower) are ignored.

5
CURL_CONNECT_TIMEOUT =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

cURL's default --connect-timeout value can be up to two minutes, so we need to use a more reasonable duration (in seconds) to avoid a lengthy wait when a connection can't be established.

10
CURL_MAX_TIME =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

cURL does not set a default --max-time value, so we provide a value to ensure cURL will time out in a reasonable amount of time.

T.let(CURL_CONNECT_TIMEOUT + 5, Integer)
CURL_PROCESS_TIMEOUT =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

The curl process will sometimes hang indefinitely (despite setting the --max-time argument) and it needs to be quit for livecheck to continue. This value is used to set the timeout argument on Utils::Curl method calls in Homebrew::Livecheck::Strategy.

T.let(CURL_MAX_TIME + 5, Integer)
MAX_REDIRECTIONS =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

The maximum number of redirections that curl should allow.

5
MAX_PARSE_ITERATIONS =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

This value is passed to #parse_curl_output to ensure that the limit for the number of responses it will parse corresponds to the maximum number of responses in this context. The + 1 here accounts for the situation where there are exactly MAX_REDIRECTIONS number of redirections, followed by a final 200 OK response.

T.let(MAX_REDIRECTIONS + 1, Integer)
DEFAULT_CURL_ARGS =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

Baseline curl arguments used in Homebrew::Livecheck::Strategy methods.

T.let([
  # Follow redirections to handle mirrors, relocations, etc.
  "--location",
  "--max-redirs", MAX_REDIRECTIONS.to_s,
  # Avoid progress bar text, so we can reliably identify `curl` error
  # messages in output
  "--silent"
].freeze, T::Array[String])
PAGE_CONTENT_CURL_ARGS =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

curl arguments used in Strategy#page_content method.

T.let(([
  "--compressed",
  # Return an error when the HTTP response code is 400 or greater but
  # continue to return body content
  "--fail-with-body",
  # Include HTTP response headers in output, so we can identify the
  # final URL after any redirections
  "--include",
] + DEFAULT_CURL_ARGS).freeze, T::Array[String])
DEFAULT_CURL_OPTIONS =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

Baseline curl options used in Homebrew::Livecheck::Strategy methods.

T.let({
  print_stdout:    false,
  print_stderr:    false,
  debug:           false,
  verbose:         false,
  timeout:         CURL_PROCESS_TIMEOUT,
  connect_timeout: CURL_CONNECT_TIMEOUT,
  max_time:        CURL_MAX_TIME,
  retries:         0,
}.freeze, T::Hash[Symbol, T.untyped])
TARBALL_EXTENSION_REGEX =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

A regex used to identify a tarball extension at the end of a string.

/
  \.t
  (?:ar(?:\.(?:bz2|gz|lz|lzma|lzo|xz|Z|zst))?|
  b2|bz2?|z2|az|gz|lz|lzma|xz|Z|aZ|zst)
  $
/ix
INVALID_BLOCK_RETURN_VALUE_MSG =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

An error message to use when a strategy block returns a value of an inappropriate type.

"Return value of a strategy block must be a string or array of strings."

Class Method Summary collapse

Methods included from Utils::Curl

clear_path_cache, curl, curl_args, curl_check_http_content, curl_download, curl_executable, curl_headers, curl_http_content_headers_and_checksum, curl_output, curl_path, curl_response_follow_redirections, curl_response_last_location, curl_supports_fail_with_body?, curl_supports_tls13?, curl_version, curl_with_workarounds, http_status_ok?, parse_curl_output, url_protected_by_cloudflare?, url_protected_by_incapsula?

Methods included from SystemCommand::Mixin

#system_command, #system_command!

Class Method Details

.from_symbol(symbol) ⇒ T.untyped

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns the strategy that corresponds to the provided Symbol (or nil if there is no matching strategy).

Parameters:

  • symbol (Symbol, nil) —

    the strategy name in snake case as a Symbol (e.g. :page_match)

Returns:

  • (T.untyped) 


118
119
120
 
# File 'livecheck/strategy.rb', line 118

def self.from_symbol(symbol)
  strategies[symbol] if symbol.present?
end

.from_url(url, livecheck_strategy: nil, regex_provided: false, block_provided: false) ⇒ Array<T.untyped>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns an array of strategies that apply to the provided URL.

Parameters:

  • url (String) —

    the URL to check for matching strategies

  • livecheck_strategy (Symbol, nil) (defaults to: nil) —

    a strategy symbol from the livecheck block

  • regex_provided (Boolean) (defaults to: false) —

    whether a regex is provided in the livecheck block

  • block_provided (Boolean) (defaults to: false) —

    whether a strategy block is provided in the livecheck block

Returns:



140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
 
# File 'livecheck/strategy.rb', line 140

def self.from_url(url, livecheck_strategy: nil, regex_provided: false, block_provided: false)
  usable_strategies = strategies.select do |strategy_symbol, strategy|
    if strategy == PageMatch
      # Only treat the strategy as usable if the `livecheck` block
      # contains a regex and/or `strategy` block
      next if !regex_provided && !block_provided
    elsif [Json, Xml, Yaml].include?(strategy)
      # Only treat the strategy as usable if the `livecheck` block
      # specifies the strategy and contains a `strategy` block
      next if (livecheck_strategy != strategy_symbol) || !block_provided
    elsif strategy.const_defined?(:PRIORITY) &&
          !strategy.const_get(:PRIORITY).positive? &&
          livecheck_strategy != strategy_symbol
      # Ignore strategies with a priority of 0 or lower, unless the
      # strategy is specified in the `livecheck` block
      next
    end

    strategy.respond_to?(:match?) && strategy.match?(url)
  end.values

  # Sort usable strategies in descending order by priority, using the
  # DEFAULT_PRIORITY when a strategy doesn't contain a PRIORITY constant
  usable_strategies.sort_by do |strategy|
    (strategy.const_defined?(:PRIORITY) ? -strategy.const_get(:PRIORITY) : -DEFAULT_PRIORITY)
  end
end

.handle_block_return(value) ⇒ Array<String>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Handles the return value from a strategy block in a livecheck block.

Parameters:

  • value (T.untyped) —

    [] the return value from a strategy block

Returns:



293
294
295
296
297
298
299
300
301
302
303
304
 
# File 'livecheck/strategy.rb', line 293

def self.handle_block_return(value)
  case value
  when String
    [value]
  when Array
    value.compact.uniq
  when nil
    []
  else
    raise TypeError, INVALID_BLOCK_RETURN_VALUE_MSG
  end
end

.page_content(url, options: Options.new) ⇒ Hash{Symbol => T.untyped}

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Fetches the content at the URL and returns a hash containing the content and, if there are any redirections, the final URL. If curl encounters an error, the hash will contain a :messages array with the error message instead.

Parameters:

  • url (String) —

    the URL of the content to check

  • options (Options) (defaults to: Options.new) —

    options to modify behavior

Returns:



248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
 
# File 'livecheck/strategy.rb', line 248

def self.page_content(url, options: Options.new)
  if options.post_form || options.post_json
    curl_post_args = ["--request", "POST", *post_args(
      post_form: options.post_form,
      post_json: options.post_json,
    )]
  end

  stderr = T.let(nil, T.nilable(String))
  [:default, :browser].each do |user_agent|
    stdout, stderr, status = curl_output(
      *curl_post_args,
      *PAGE_CONTENT_CURL_ARGS, url,
      **DEFAULT_CURL_OPTIONS,
      use_homebrew_curl: options.homebrew_curl ||
                         !curl_supports_fail_with_body? ||
                         false,
      user_agent:
    )
    next unless status.success?

    # stdout contains the header information followed by the page content.
    # We use #scrub here to avoid "invalid byte sequence in UTF-8" errors.
    output = stdout.scrub

    # Separate the head(s)/body and identify the final URL (after any
    # redirections)
    parsed_output = parse_curl_output(output, max_iterations: MAX_PARSE_ITERATIONS)
    final_url = curl_response_last_location(parsed_output[:responses], absolutize: true, base_url: url)

    data = { content: parsed_output[:body] }
    data[:final_url] = final_url if final_url.present? && final_url != url
    return data
  end

  error_msgs = stderr&.scan(/^curl:.+$/)
  { messages: error_msgs.presence || ["cURL failed without a detectable error"] }
end

.page_headers(url, options: Options.new) ⇒ Array<Hash{String => String}>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Collects HTTP response headers, starting with the provided URL. Redirections will be followed and all the response headers are collected into an array of hashes.

Parameters:

  • url (String) —

    the URL to fetch

  • options (Options) (defaults to: Options.new) —

    options to modify behavior

Returns:



206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
 
# File 'livecheck/strategy.rb', line 206

def self.page_headers(url, options: Options.new)
  headers = []

  if options.post_form || options.post_json
    curl_post_args = ["--request", "POST", *post_args(
      post_form: options.post_form,
      post_json: options.post_json,
    )]
  end

  [:default, :browser].each do |user_agent|
    begin
      parsed_output = curl_headers(
        *curl_post_args,
        "--max-redirs",
        MAX_REDIRECTIONS.to_s,
        url,
        wanted_headers:    ["location", "content-disposition"],
        use_homebrew_curl: options.homebrew_curl || false,
        user_agent:,
        **DEFAULT_CURL_OPTIONS,
      )
    rescue ErrorDuringExecution
      next
    end

    parsed_output[:responses].each { |response| headers << response[:headers] }
    break if headers.present?
  end

  headers
end

.post_args(post_form: nil, post_json: nil) ⇒ Array<String>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Creates curl --data or --json arguments (for POST requests) from relatedlivecheckblockurl` options.

Parameters:

  • post_form (Hash{Symbol => String}, nil) (defaults to: nil) —

    data to encode using URI::encode_www_form

  • post_json (Hash{Symbol => T.anything}, nil) (defaults to: nil) —

    data to encode using JSON::generate

Returns:



180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
 
# File 'livecheck/strategy.rb', line 180

def self.post_args(post_form: nil, post_json: nil)
  args = if post_form.present?
    require "uri"
    ["--data", URI.encode_www_form(post_form)]
  elsif post_json.present?
    require "json"
    ["--json", JSON.generate(post_json)]
  else
    []
  end

  if (content_length = args[1]&.length)
    args << "--header" << "Content-Length: #{content_length}"
  end

  args
end