Change in PURL since syft v.1.19.0

TimBrown1611 · January 26, 2025, 6:51am

Hi team,

I was comparing output of syft, and saw some changed i purl of packages in debian.

can you please confirm this is the desired behave?

Before:
"purl": "pkg:deb/debian/zlib1g@1:1.2.7.dfsg-13?arch=amd64&distro=debian-7&upstream=zlib"

After:
"purl": "pkg:deb/debian/zlib1g@1**%3A**1.2.7.dfsg-13?arch=amd64&distro=debian-7&upstream=zlib"

the image is -
m3tr0x/avivm-sushi:0.0.2

(I saw few more packages with the same change)

Thanks!

willmurphy · January 27, 2025, 3:22pm

Hi @TimBrown1611 thanks for the post.

Can you confirm the difference you’re seeing?

I see the following diff (repeated on basically every deb):

-"pkg:deb/debian/zlib1g@1:1.2.7.dfsg-13?arch=amd64&distro=debian-7&upstream=zlib"
+"pkg:deb/debian/zlib1g@1%3A1.2.7.dfsg-13?arch=amd64&distro=debian-7&upstream=zlib"

where the only change is that the : in between the epoch and the rest of the version is now percent-encoded. However, in your example, there are some ** in the diff, as in zlib1g@1**%3A**1.2.7.dfsg-13. Can you confirm that you’re really seeing these **, or is there some redaction or abbreviation in what you posted? (The percent encoding worries me less than the **.)

TimBrown1611 · January 27, 2025, 3:33pm

Hi!
not the ** (it is to make it bold)

the %3A is the diff

willmurphy · January 27, 2025, 3:37pm

Thanks for getting back to me so quickly @TimBrown1611!

The change you are seeing is because of Escape each segment according to spec by jkugler · Pull Request #23 · anchore/packageurl-go · GitHub

I believe this change is intentional and correct. Is it causing any issues for you?

TimBrown1611 · January 27, 2025, 3:38pm

not an issue that I’ve noticed, just wanted to make sure it is OK

willmurphy · January 27, 2025, 3:52pm

Checking the PURL spec,

It says: purl-spec/PURL-SPECIFICATION.rst at 1951d217bde29590a73f075db4ab71cc00011459 · package-url/purl-spec · GitHub

A version must be a percent-encoded string.

But at purl-spec/PURL-TYPES.rst at 1951d217bde29590a73f075db4ab71cc00011459 · package-url/purl-spec · GitHub

It gives the example pkg:deb/debian/attr@1:2.4.47-2%2Bb1?arch=amd64 which percent encodes + but not :, in the version, which is pretty confusing.

@Christopher_Phillips did this come up at all when discussing this? I feel like the PURL spec is trying to tell me 2 different things.

For RPMs, the epoch is a qualifier (see purl-spec/PURL-TYPES.rst at 1951d217bde29590a73f075db4ab71cc00011459 · package-url/purl-spec · GitHub), so we shouldn’t have : in the version string, so RPMs shouldn’t be changing I think.

@popey do debs have an epoch? Or do they use a different term for the bit of the version before the :? Does it mean something different than with RPMs?

spiffcs · January 27, 2025, 4:19pm

Yes! This question did come up when we were discussing the change with the user who submitted the PR, but only with regards to the name component not version.

You’re correct that the documentation from their seems to convey two different things. PURL TYPES has examples where a version component is not percent encoded, but the PURL-SPECIFICATION.rst says

A version must be a percent-encoded string

The conclusion on the PR/issues was that If we produce a name or namespace that is not percent encoded we are in violation of the spec. I think the same logic would apply to version here.

Maybe we should ask about this seeming conflict for the version component that we see between PURL-TYPES and PURL-SPECIFICATION

We opted to follow PURL-SPECIFICATION regarding the name component so that our behavior would be equivalent with the upstream packageurl-go library which percent encodes purl components in this way:

github.com/package-url/packageurl-go

packageurl.go

4b484f71a


      
          // escape the given string in a purl-compatible way.
          func escape(s string) string {
          	// for compatibility with other implementations and the purl-spec, we want to escape all
          	// characters, which is what "QueryEscape" does. The issue with QueryEscape is that it encodes
          	// " " (space) as "+", which is valid in a query, but invalid in a path (see
          	// https://stackoverflow.com/questions/2678551/when-should-space-be-encoded-to-plus-or-20) for
          	// context).
          	// To work around that, we replace the "+" signs with the path-compatible "%20".
          	return strings.ReplaceAll(url.QueryEscape(s), "+", "%20")
          }

Our PURL library has a couple of other changes that are ahead of main in the upstream library which is why we opt to keep the fork.

In this comment we called out the change under the heading Test to look at that's actually related to the code change Note this was with regards to / under namespace, but I believe the same logic of why we went with percent encoded / in namespace can be applied to which we percent encode : in the name parameter.

github.com/anchore/packageurl-go

Comment by spiffcs - Escape each segment according to spec

anchore:master ← jkugler:fix_escaping_method_to_comply_with_spec

Yea I'm a little stumped for why the upstream test data is failing and will talk… with the team a bit about adding purl type specific validation logic after this PR is done and satisfying these tests for CI purposes. I didn't see anything related to cpan/distribution validation in the spec. It just seems to be validation for a specific type that was added into the test cases. # Yak shaving tests to resolve for CI purposes Validation for the cpan type: https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst#cpan ``` cpan module name like distribution name did not fail and returned packageurl.PackageURL ``` I tried to get out of the golang library bias and check another library for the truth regarding the failing test cases, but even they don't do this kind of validation. https://github.com/package-url/packageurl-python ``` >>> from packageurl import PackageURL >>> purl = PackageURL.from_string("pkg:cpan/Perl-Version@1.013") >>> purl PackageURL(type='cpan', namespace=None, name='Perl-Version', version='1.013', qualifiers={}, subpath=None) >>> ``` But the test data(which we pull from [here](https://raw.githubusercontent.com/package-url/purl-spec/master/test-suite-data.json)) says that this should be invalid: ``` { "description": "cpan module name like distribution name", "purl": "pkg:cpan/Perl-Version@1.013", "canonical_purl": "pkg:cpan/Perl-Version@1.013", "type": "cpan", "namespace": null, "name": "Perl-Version", "version": "1.013", "qualifiers": null, "subpath": null, "is_invalid": true }, ``` Another case with the same description passing validation and being constructed ``` >>> purl = PackageURL.from_string("pkg:cpan/GDT/URI::PackageURL@2.11") >>> purl PackageURL(type='cpan', namespace='GDT', name='URI::PackageURL', version='2.11', qualifiers={}, subpath=None) ``` I proved this library does have validation by trying an obviously invalid PURL: ``` purl = PackageURL.from_string("pkg:generic/@package@name@@1.0.0?invalid&param") Traceback (most recent call last): File "<python-input-6>", line 1, in <module> purl = PackageURL.from_string("pkg:generic/@package@name@@1.0.0?invalid&param") File "/Users/hal/Library/Caches/pypoetry/virtualenvs/vunnel-wv_ZmzTK-py3.13/lib/python3.13/site-packages/packageurl/__init__.py", line 539, in from_string type, namespace, name, version, qualifiers, subpath = normalize( # NOQA ``` # Test to look at that's actually related to the code change I think one failing test that's worth highlighting with this change is `TestNameEscaping` This is a difference in our fork from the upstream packageurl-go. With the code change as is the following example is getting encoded from `ab/c` --> `ab%2Fc` I'll take a stab at integrating your fix so that this test with the current code so that this case will pass. The test case covers: https://github.com/package-url/purl-spec/blob/1951d217bde29590a73f075db4ab71cc00011459/PURL-SPECIFICATION.rst?plain=1#L253-L255 Test output for https://github.com/anchore/packageurl-go/blob/5c22e6360c4f22f7910d3e3ef8cd34180502e480/packageurl_test.go#L312-L329 ``` wrong escape. expected="pkg:deb/ab/c", got="pkg:deb/ab%2Fc" ```

Here is the discussion from live stream:

It’s also worth mentioning that previous versions of syft using the old behavior would generate PURL that failed on submission to other PURL library with a : not percent encoded in the name parameter.

popey · January 27, 2025, 4:50pm

Yup!

Here’s the version section of the debian packaging guide.

5.6.12. Version

The version number of a package. The format is: [epoch:]upstream_version[-debian_revision].

The three components here are:

epoch

This is a single (generally small) unsigned integer. It may be omitted, in which case zero is assumed.

Epochs can help when the upstream version numbering scheme changes, but they must be used with care. You should not change the epoch, even in experimental, without getting consensus on debian-devel first.

Topic		Replies	Views
Syft - v1.16.0 released Announcements release	1	15	January 16, 2025
Syft - v1.10.0 released Announcements	1	14	January 16, 2025
Syft - v1.18.0 released Announcements release	1	8	January 16, 2025
Stdlib versions Syft	2	19	December 31, 2024
Syft - v1.12.2 released Announcements release	1	10	January 16, 2025

Change in PURL since syft v.1.19.0

Related topics