Module: PyPI Private
- Defined in:
- utils/pypi.rb
Overview
This module is part of a private API. This module may only be used in the Homebrew/brew repository. Third parties should avoid using this module if possible, as it may be removed or changed without warning.
Helper functions for updating PyPI resources.
Defined Under Namespace
Classes: Package
Class Method Summary collapse
-
.normalize_python_package(name) ⇒ Object
private
-
.pip_report(packages) ⇒ Object
private
-
.pip_report_to_packages(report) ⇒ Object
private
-
.update_pypi_url(url, version) ⇒ String?
private
-
.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil, exclude_packages: nil, print_only: false, silent: false, ignore_non_pypi_packages: false) ⇒ Boolean?
private
Return true if resources were checked (even if no change).
Class Method Details
.normalize_python_package(name) ⇒ Object
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
345 346 347 348 349 |
# File 'utils/pypi.rb', line 345 def self.normalize_python_package(name) # This normalization is defined in the PyPA packaging specifications; # https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization name.gsub(/[-_.]+/, "-").downcase end |
.pip_report(packages) ⇒ Object
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 |
# File 'utils/pypi.rb', line 351 def self.pip_report(packages) return [] if packages.blank? command = [Formula["python"].bin/"python3", "-m", "pip", "install", "-q", "--dry-run", "--ignore-installed", "--report=/dev/stdout", *packages.map(&:to_s)] pip_output = Utils.popen_read({ "PIP_REQUIRE_VIRTUALENV" => "false" }, *command) unless $CHILD_STATUS.success? odie <<~EOS Unable to determine dependencies for "#{packages.join(" ")}" because of a failure when running `#{command.join(" ")}`. Please update the resources manually. EOS end pip_report_to_packages(JSON.parse(pip_output)).uniq end |
.pip_report_to_packages(report) ⇒ Object
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
367 368 369 370 371 372 373 374 375 376 |
# File 'utils/pypi.rb', line 367 def self.pip_report_to_packages(report) return [] if report.blank? report["install"].map do |package| name = normalize_python_package(package["metadata"]["name"]) version = package["metadata"]["version"] Package.new "#{name}==#{version}" end.compact end |
.update_pypi_url(url, version) ⇒ String?
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
181 182 183 184 185 186 187 188 189 190 |
# File 'utils/pypi.rb', line 181 def self.update_pypi_url(url, version) package = Package.new url, is_url: true return unless package.valid_pypi_package? _, url = package.pypi_info(new_version: version) url rescue ArgumentError nil end |
.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil, exclude_packages: nil, print_only: false, silent: false, ignore_non_pypi_packages: false) ⇒ Boolean?
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
Return true if resources were checked (even if no change).
205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 |
# File 'utils/pypi.rb', line 205 def self.update_python_resources!(formula, version: nil, package_name: nil, extra_packages: nil, exclude_packages: nil, print_only: false, silent: false, ignore_non_pypi_packages: false) auto_update_list = formula.tap&.pypi_formula_mappings if auto_update_list.present? && auto_update_list.key?(formula.full_name) && package_name.blank? && extra_packages.blank? && exclude_packages.blank? list_entry = auto_update_list[formula.full_name] case list_entry when false unless print_only odie "The resources for \"#{formula.name}\" need special attention. Please update them manually." end when String package_name = list_entry when Hash package_name = list_entry["package_name"] extra_packages = list_entry["extra_packages"] exclude_packages = list_entry["exclude_packages"] end end main_package = if package_name.present? Package.new(package_name) else stable = T.must(formula.stable) url = if stable.specs[:tag].present? url = "git+#{stable.url}@#{stable.specs[:tag]}" else stable.url end Package.new(url, is_url: true) end if version.present? if main_package.valid_pypi_package? main_package.version = version else return if ignore_non_pypi_packages odie "The main package is not a PyPI package, meaning that version-only updates cannot be \ performed. Please update its URL manually." end end extra_packages = (extra_packages || []).map { |p| Package.new p } exclude_packages = (exclude_packages || []).map { |p| Package.new p } exclude_packages += %w[argparse pip setuptools wsgiref].map { |p| Package.new p } # remove packages from the exclude list if we've explicitly requested them as an extra package exclude_packages.delete_if { |package| extra_packages.include?(package) } input_packages = [main_package] extra_packages.each do |extra_package| if !extra_package.valid_pypi_package? && !ignore_non_pypi_packages odie "\"#{extra_package}\" is not available on PyPI." end input_packages.each do |existing_package| if existing_package.same_package?(extra_package) && existing_package.version != extra_package.version odie "Conflicting versions specified for the `#{extra_package.name}` package: " \ "#{existing_package.version}, #{extra_package.version}" end end input_packages << extra_package unless input_packages.include? extra_package end formula.resources.each do |resource| if !print_only && !resource.url.start_with?(PYTHONHOSTED_URL_PREFIX) odie "\"#{formula.name}\" contains non-PyPI resources. Please update the resources manually." end end ensure_formula_installed!("python") # Resolve the dependency tree of all input packages ohai "Retrieving PyPI dependencies for \"#{input_packages.join(" ")}\"..." if !print_only && !silent found_packages = pip_report(input_packages) # Resolve the dependency tree of excluded packages to prune the above exclude_packages.delete_if { |package| found_packages.exclude? package } ohai "Retrieving PyPI dependencies for excluded \"#{exclude_packages.join(" ")}\"..." if !print_only && !silent exclude_packages = pip_report(exclude_packages) + [Package.new(main_package.name)] new_resource_blocks = "" found_packages.sort.each do |package| if exclude_packages.include? package ohai "Excluding \"#{package}\"" if !print_only && !silent next end ohai "Getting PyPI info for \"#{package}\"" if !print_only && !silent name, url, checksum = package.pypi_info # Fail if unable to find name, url or checksum for any resource if name.blank? odie "Unable to resolve some dependencies. Please update the resources for \"#{formula.name}\" manually." elsif url.blank? || checksum.blank? odie <<~EOS Unable to find the URL and/or sha256 for the "#{name}" resource. Please update the resources for "#{formula.name}" manually. EOS end # Append indented resource block new_resource_blocks += <<-EOS resource "#{name}" do url "#{url}" sha256 "#{checksum}" end EOS end if print_only puts new_resource_blocks.chomp return end # Check whether resources already exist (excluding virtualenv dependencies) if formula.resources.all? { |resource| resource.name.start_with?("homebrew-") } # Place resources above install method inreplace_regex = / def install/ new_resource_blocks += " def install" else # Replace existing resource blocks with new resource blocks inreplace_regex = / (resource .* do\s+url .*\s+sha256 .*\s+ end\s*)+/ new_resource_blocks += " " end ohai "Updating resource blocks" unless silent Utils::Inreplace.inreplace formula.path do |s| if s.inreplace_string.scan(inreplace_regex).length > 1 odie "Unable to update resource blocks for \"#{formula.name}\" automatically. Please update them manually." end s.sub! inreplace_regex, new_resource_blocks end true end |