Module: PyPI — Homebrew Ruby API

.normalize_python_package(name) ⇒ `String`

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

name (String)

Returns:

(String)

# File 'utils/pypi.rb', line 453

def self.normalize_python_package(name)
  # This normalization is defined in the PyPA packaging specifications;
  # https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization
  name.gsub(/[-_.]+/, "-").downcase
end

.pip_report(packages, python_name: "python", print_stderr: false) ⇒ `Array<Package>`

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

packages (Array<Package>)
python_name (String) (defaults to: "python")
print_stderr (Boolean) (defaults to: false)

Returns:

(Array<Package>)

# File 'utils/pypi.rb', line 464

def self.pip_report(packages, python_name: "python", print_stderr: false)
  return [] if packages.blank?

  command = [
    Formula[python_name].opt_libexec/"bin/python", "-m", "pip", "install", "-q", "--disable-pip-version-check",
    "--dry-run", "--ignore-installed", "--report=/dev/stdout", *packages.map(&:to_s)
  ]
  options = {}
  options[:err] = :err if print_stderr
  pip_output = Utils.popen_read({ "PIP_REQUIRE_VIRTUALENV" => "false" }, *command, **options)
  unless $CHILD_STATUS.success?
    odie <<~EOS
      Unable to determine dependencies for "#{packages.join(" ")}" because of a failure when running
      `#{command.join(" ")}`.
      Please update the resources manually.
    EOS
  end
  pip_report_to_packages(JSON.parse(pip_output)).uniq
end

.pip_report_to_packages(report) ⇒ `Array<Package>`

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

report (Hash{String => T.untyped})

Returns:

(Array<Package>)

# File 'utils/pypi.rb', line 485

def self.pip_report_to_packages(report)
  return [] if report.blank?

  report["install"].filter_map do |package|
    name = normalize_python_package(package["metadata"]["name"])
    version = package["metadata"]["version"]

    Package.new "#{name}==#{version}"
  end
end

.update_pypi_url(url, version) ⇒ `String`^?

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

url (String)
version (String, Version)

Returns:

(String, nil)

# File 'utils/pypi.rb', line 202

def self.update_pypi_url(url, version)
  package = Package.new url, is_url: true

  return unless package.valid_pypi_package?

  _, url = package.pypi_info(new_version: version)
  url
rescue ArgumentError
  nil
end

.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil, exclude_packages: nil, dependencies: nil, install_dependencies: false, print_only: false, silent: false, verbose: false, ignore_errors: false, ignore_non_pypi_packages: false) ⇒ `Boolean`^?

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Return true if resources were checked (even if no change).

Parameters:

formula (Formula)
version (String, nil) (defaults to: nil)
package_name (String, nil) (defaults to: nil)
extra_packages (Array<String>, nil) (defaults to: nil)
exclude_packages (Array<String>, nil) (defaults to: nil)
dependencies (Array<String>, nil) (defaults to: nil)
install_dependencies (Boolean, nil) (defaults to: false)
print_only (Boolean, nil) (defaults to: false)
silent (Boolean, nil) (defaults to: false)
verbose (Boolean, nil) (defaults to: false)
ignore_errors (Boolean, nil) (defaults to: false)
ignore_non_pypi_packages (Boolean, nil) (defaults to: false)

Returns:

(Boolean, nil)

# File 'utils/pypi.rb', line 230

def self.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil,
                                  exclude_packages: nil, dependencies: nil, install_dependencies: false,
                                  print_only: false, silent: false, verbose: false,
                                  ignore_errors: false, ignore_non_pypi_packages: false)
  auto_update_list = formula.tap&.pypi_formula_mappings
  if auto_update_list.present? && auto_update_list.key?(formula.full_name) &&
     package_name.blank? && extra_packages.blank? && exclude_packages.blank?

    list_entry = auto_update_list[formula.full_name]
    case list_entry
    when false
      unless print_only
        odie "The resources for \"#{formula.name}\" need special attention. Please update them manually."
      end
    when String
      package_name = list_entry
    when Hash
      package_name = list_entry["package_name"]
      extra_packages = list_entry["extra_packages"]
      exclude_packages = list_entry["exclude_packages"]
      dependencies = list_entry["dependencies"]
    end
  end

  missing_dependencies = Array(dependencies).reject do |dependency|
    Formula[dependency].any_version_installed?
  rescue FormulaUnavailableError
    odie "Formula \"#{dependency}\" not found but it is a dependency to update \"#{formula.name}\" resources."
  end
  if missing_dependencies.present?
    missing_msg = "formulae required to update \"#{formula.name}\" resources: #{missing_dependencies.join(", ")}"
    odie "Missing #{missing_msg}" unless install_dependencies
    ohai "Installing #{missing_msg}"
    missing_dependencies.each(&:ensure_formula_installed!)
  end

  python_deps = formula.deps
                       .select { |d| d.name.match?(/^python(@.+)?$/) }
                       .map(&:to_formula)
                       .sort_by(&:version)
                       .reverse
  python_name = if python_deps.empty?
    "python"
  else
    (python_deps.find(&:any_version_installed?) || python_deps.first).name
  end

  main_package = if package_name.present?
    package_string = package_name
    package_string += "==#{formula.version}" if version.blank? && formula.version.present?
    Package.new(package_string, python_name:)
  elsif package_name == ""
    nil
  else
    stable = T.must(formula.stable)
    url = if stable.specs[:tag].present?
      "git+#{stable.url}@#{stable.specs[:tag]}"
    else
      stable.url
    end
    Package.new(url, is_url: true, python_name:)
  end

  if main_package.nil?
    odie "The main package was skipped but no PyPI `extra_packages` were provided." if extra_packages.blank?
  elsif version.present?
    if main_package.valid_pypi_package?
      main_package.version = version
    else
      return if ignore_non_pypi_packages

      odie "The main package is not a PyPI package, meaning that version-only updates cannot be \
        performed. Please update its URL manually."
    end
  end

  extra_packages = (extra_packages || []).map { |p| Package.new p }
  exclude_packages = (exclude_packages || []).map { |p| Package.new p }
  exclude_packages += %w[argparse pip wsgiref].map { |p| Package.new p }
  if (newest_python = python_deps.first) && newest_python.version < Version.new("3.12")
    exclude_packages.append(Package.new("setuptools"))
  end
  # remove packages from the exclude list if we've explicitly requested them as an extra package
  exclude_packages.delete_if { |package| extra_packages.include?(package) }

  input_packages = Array(main_package)
  extra_packages.each do |extra_package|
    if !extra_package.valid_pypi_package? && !ignore_non_pypi_packages
      odie "\"#{extra_package}\" is not available on PyPI."
    end

    input_packages.each do |existing_package|
      if existing_package.same_package?(extra_package) && existing_package.version != extra_package.version
        odie "Conflicting versions specified for the `#{extra_package.name}` package: " \
             "#{existing_package.version}, #{extra_package.version}"
      end
    end

    input_packages << extra_package unless input_packages.include? extra_package
  end

  formula.resources.each do |resource|
    if !print_only && !resource.url.start_with?(PYTHONHOSTED_URL_PREFIX)
      odie "\"#{formula.name}\" contains non-PyPI resources. Please update the resources manually."
    end
  end

  ensure_formula_installed!(python_name)

  # Resolve the dependency tree of all input packages
  show_info = !print_only && !silent
  ohai "Retrieving PyPI dependencies for \"#{input_packages.join(" ")}\"..." if show_info

  print_stderr = if verbose && show_info
    true
  else
    false
  end

  found_packages = pip_report(input_packages, python_name:, print_stderr:)
  # Resolve the dependency tree of excluded packages to prune the above
  exclude_packages.delete_if { |package| found_packages.exclude? package }
  ohai "Retrieving PyPI dependencies for excluded \"#{exclude_packages.join(" ")}\"..." if show_info
  exclude_packages = pip_report(exclude_packages, python_name:, print_stderr:)
  if (main_package_name = main_package&.name)
    exclude_packages += [Package.new(main_package_name)]
  end

  new_resource_blocks = ""
  package_errors = ""
  found_packages.sort.each do |package|
    if exclude_packages.include? package
      ohai "Excluding \"#{package}\"" if show_info
      exclude_packages.delete package
      next
    end

    ohai "Getting PyPI info for \"#{package}\"" if show_info
    name, url, checksum, _, package_error = package.pypi_info(ignore_errors: ignore_errors)
    if package_error.blank?
      # Fail if unable to find name, url or checksum for any resource
      if name.blank?
        if ignore_errors
          package_error = "unknown failure"
        else
          odie "Unable to resolve some dependencies. Please update the resources for \"#{formula.name}\" manually."
        end
      elsif url.blank? || checksum.blank?
        if ignore_errors
          package_error = "unable to find URL and/or sha256"
        else
          odie <<~EOS
            Unable to find the URL and/or sha256 for the "#{name}" resource.
            Please update the resources for "#{formula.name}" manually.
          EOS
        end
      end
    end

    if package_error.blank?
      # Append indented resource block
      new_resource_blocks += <<-EOS
resource "#{name}" do
  url "#{url}"
  sha256 "#{checksum}"
end

      EOS
    else
      # Leave a placeholder for formula author to investigate
      package_errors += "  # RESOURCE-ERROR: Unable to resolve \"#{package}\" (#{package_error})\n"
    end
  end

  package_errors += "\n" if package_errors.present?
  resource_section = "#{package_errors}#{new_resource_blocks}"

  odie "Excluded superfluous packages: #{exclude_packages.join(", ")}" if exclude_packages.any?

  if print_only
    puts resource_section.chomp
    return
  end

  # Check whether resources already exist (excluding virtualenv dependencies)
  if formula.resources.all? { |resource| resource.name.start_with?("homebrew-") }
    # Place resources above install method
    inreplace_regex = /  def install/
    resource_section += "  def install"
  else
    # Replace existing resource blocks with new resource blocks
    inreplace_regex = /
      \ \ (
      (\#\ RESOURCE-ERROR:\ .*\s+)*
      resource\ .*\ do\s+
        url\ .*\s+
        sha256\ .*\s+
        ((\#.*\s+)*
        patch\ (.*\ )?do\s+
          url\ .*\s+
          sha256\ .*\s+
        end\s+)*
      end\s+)+
    /x
    resource_section += "  "
  end

  ohai "Updating resource blocks" unless silent
  Utils::Inreplace.inreplace formula.path do |s|
    if T.must(s.inreplace_string.split(/^  test do\b/, 2).first).scan(inreplace_regex).length > 1
      odie "Unable to update resource blocks for \"#{formula.name}\" automatically. Please update them manually."
    end
    s.sub! inreplace_regex, resource_section
  end

  if package_errors.present?
    ofail "Unable to resolve some dependencies. Please check #{formula.path} for RESOURCE-ERROR comments."
  end

  true
end

Module: PyPI Private

Overview

Defined Under Namespace

Class Method Summary collapse

Class Method Details

.normalize_python_package(name) ⇒ `String`

.pip_report(packages, python_name: "python", print_stderr: false) ⇒ `Array<Package>`

.pip_report_to_packages(report) ⇒ `Array<Package>`

.update_pypi_url(url, version) ⇒ `String`^?

.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil, exclude_packages: nil, dependencies: nil, install_dependencies: false, print_only: false, silent: false, verbose: false, ignore_errors: false, ignore_non_pypi_packages: false) ⇒ `Boolean`^?

Module: PyPI Private

Overview

Defined Under Namespace

Class Method Summary collapse

Class Method Details

.normalize_python_package(name) ⇒ String

.pip_report(packages, python_name: "python", print_stderr: false) ⇒ Array<Package>

.pip_report_to_packages(report) ⇒ Array<Package>

.update_pypi_url(url, version) ⇒ String?

.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil, exclude_packages: nil, dependencies: nil, install_dependencies: false, print_only: false, silent: false, verbose: false, ignore_errors: false, ignore_non_pypi_packages: false) ⇒ Boolean?

.normalize_python_package(name) ⇒ `String`

.pip_report(packages, python_name: "python", print_stderr: false) ⇒ `Array<Package>`

.pip_report_to_packages(report) ⇒ `Array<Package>`

.update_pypi_url(url, version) ⇒ `String`^?

.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil, exclude_packages: nil, dependencies: nil, install_dependencies: false, print_only: false, silent: false, verbose: false, ignore_errors: false, ignore_non_pypi_packages: false) ⇒ `Boolean`^?