Syft process getting hung

I am using version syft version 1.14.0. It is working well for most of the artifacts. But we have a use case to scan 50 GB directory. Syft actually worked scanning more than 200+ GB directory but it is getting hung for couple of 50+ GB directories.

I tried verifying if there are any infinite symlinks/loops. But it did not contain any.

I tried using strace. It shows below output

futex(0x2e68160, FUTEX_WAIT_PRIVATE, 0, NULL

I tried taking thread dump (Goroutine dump). Detailed go routine logs here

Hi @santhosh thanks for the post!

Sorry to hear Syft is hanging on one of your directory scans. It’s hard to know exactly what’s going, but I have a few guesses/questions that might help.

One thing I saw in the goroutine dump that you linked is that Syft has a couple goroutines that are trying to dereference pom.xml properties in the Java cataloger. Do you know whether there might be a pom.xml property that forms a cycle defining one pom.xml property in terms of another, then the other in terms of the first?

Another thing that might make where Syft is hanging more obvious is some debug logs. Can you re-run Syft with SYFT_LOG_FILE set to some path and -v to get info logs (or -vv for debug or -vvv for trace)? That will make quite a large log file on a 50GB directory, but it might make it obvious where Syft is getting stuck, since there it will either give us a log line after which Syft prints nothing, or, hopefully, give use a series of log lines that get printed over and over again indicating where Syft is caught in a cycle.

Another thing you can try is turning different catalogers on or off, to try to see which cataloger is hanging. If there’s a cataloger that you can turn off, that makes the scan finish, that might be an indication that Syft is hanging in that cataloger. You can read about how to select catalogers at Package Cataloger Selection · anchore/syft Wiki · GitHub - if turning off the Java catalogers makes the scan finish quickly, that might be more evidence that Syft is getting stuck dereferencing cyclically defined pom.xml properties.

I would also be curious if you are running Syft with default configuration when it hangs.

Let me know if any of that helps! Thanks!

@willmurphy Thanks for your quick response. I am already running syft in trace mode. This 50 GB directory contains thousands of jar. so, unable to understand which jar is fault. For sure, as per log it could be related to “java-pom-cataloger”.

I tried increasing memory, CPU and also verified file descriptor limits. Still not solved. I did try using latest syft version too.

These are env variables i am using other than default values

SYFT_FILE_METADATA_CATALOGER_ENABLED=true,SYFT_CHECK_FOR_APP_UPDATE=false,SYFT_LOG_LEVEL=trace,SYFT_LOG_FILE=/tmp/syft_custom_debug.log

Below are last 100 lines in the debug log

[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=/conanfile.txt
[0665] TRACE searching filetree by glob glob=
/conanfile.txt
[0665] TRACE searching for paths matching glob glob=/conan.lock
[0665] TRACE searching filetree by glob glob=
/conan.lock
[0665] DEBUG discovered 0 packages cataloger=conan-cataloger
[0665] TRACE package cataloger completed name=conan-cataloger
[0665] TRACE starting package cataloger name=dart-pubspec-lock-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=/pubspec.lock
[0665] TRACE searching filetree by glob glob=
/pubspec.lock
[0665] DEBUG discovered 0 packages cataloger=dart-pubspec-lock-cataloger
[0665] TRACE package cataloger completed name=dart-pubspec-lock-cataloger
[0665] TRACE starting package cataloger name=dotnet-deps-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=/*.deps.json
[0665] TRACE searching filetree by glob glob=
/.deps.json
[0665] TRACE parsing file contents path=/sharedfs/fa/cas_mw_other.tar.gz_extracted/fa/APPLTOP/fmw/oit/redist/coreapp/OITRedirectSample.deps.json
[0665] TRACE parsing file contents path=/sharedfs/fa/cas_mw_other.tar.gz_extracted/fa/APPLTOP/fmw/oit/redist/coreapp/OITSample.deps.json
[0665] TRACE parsing file contents path=/sharedfs/fa/cas_mw_other.tar.gz_extracted/fa/viewroot/APPLTOP/fmw/fmw_20.07_ccv1_view/oit/redist/coreapp/OITRedirectSample.deps.json
[0665] TRACE parsing file contents path=/sharedfs/fa/cas_mw_other.tar.gz_extracted/fa/viewroot/APPLTOP/fmw/fmw_20.07_ccv1_view/oit/redist/coreapp/OITSample.deps.json
[0665] DEBUG discovered 16 packages cataloger=dotnet-deps-cataloger
[0665] TRACE package cataloger completed name=dotnet-deps-cataloger
[0665] TRACE starting package cataloger name=elixir-mix-lock-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=/mix.lock
[0665] TRACE searching filetree by glob glob=
/mix.lock
[0665] DEBUG discovered 0 packages cataloger=elixir-mix-lock-cataloger
[0665] TRACE package cataloger completed name=elixir-mix-lock-cataloger
[0665] TRACE starting package cataloger name=erlang-rebar-lock-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=/rebar.lock
[0665] TRACE searching filetree by glob glob=
/rebar.lock
[0665] DEBUG discovered 0 packages cataloger=erlang-rebar-lock-cataloger
[0665] TRACE package cataloger completed name=erlang-rebar-lock-cataloger
[0665] TRACE starting package cataloger name=erlang-otp-application-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=**/
.app
[0665] TRACE searching filetree by glob glob=/*.app
[0665] DEBUG discovered 0 packages cataloger=erlang-otp-application-cataloger
[0665] TRACE package cataloger completed name=erlang-otp-application-cataloger
[0665] TRACE starting package cataloger name=haskell-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=
/stack.yaml
[0665] TRACE searching filetree by glob glob=/stack.yaml
[0665] TRACE searching for paths matching glob glob=
/stack.yaml.lock
[0665] TRACE searching filetree by glob glob=/stack.yaml.lock
[0665] TRACE searching for paths matching glob glob=
/cabal.project.freeze
[0665] TRACE searching filetree by glob glob=/cabal.project.freeze
[0665] DEBUG discovered 0 packages cataloger=haskell-cataloger
[0665] TRACE package cataloger completed name=haskell-cataloger
[0665] TRACE starting package cataloger name=go-module-file-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=
/go.mod
[0665] TRACE searching filetree by glob glob=/go.mod
[0665] DEBUG discovered 0 packages cataloger=go-module-file-cataloger
[0665] TRACE package cataloger completed name=go-module-file-cataloger
[0665] TRACE starting package cataloger name=java-gradle-lockfile-cataloger
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/usr/lib/os-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/system-release-cpe
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/etc/redhat-release
[0665] TRACE searching filetree by path path=/tmp/scratch/2c64b6da-3704-4c0f-8049-a9a1eb9405f6/catalog/axzreuakkdwb_slaps_integration_FRE_TEST/20241410155107/STARTER_TEMPLATE_FA_BINARY.tar.gz_extracted/bin/busybox
[0665] TRACE searching for paths matching glob glob=
/gradle.lockfile*
[0665] TRACE searching filetree by glob glob=/gradle.lockfile*
[0666] DEBUG discovered 0 packages cataloger=java-gradle-lockfile-cataloger
[0666] TRACE package cataloger completed name=java-gradle-lockfile-cataloger
[0666] TRACE starting package cataloger name=java-pom-cataloger
[0666] TRACE searching filetree by glob glob=
/pom.xml
[0666] DEBUG error resolving maven property error=unable to resolve pom com.oracle.weblogic.archetype wls-common 14.1.2-0-0: %!w() propertyValue=${groupId}
[0666] DEBUG error resolving maven property error=unable to resolve pom com.oracle.weblogic.archetype wls-common 14.1.2-0-0: %!w() propertyValue=${artifactId}
[0666] DEBUG error resolving maven property error=unable to resolve pom com.oracle.weblogic.archetype wls-common 14.1.2-0-0: %!w() propertyValue=${version}
{code}

@willmurphy I found the the file where syft is haning. It is haning on one pom file. I was confused why it was hanging. Pom is so simple that it contains only one dependency.

Also, it is resolved after changing “max-parent-recursive-depth” to 1. We were initially not setting any value. Also, our system is disconnected from internet. So, thinking that there might be a bug in Syft where it might be hung when internet is not available.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>${groupId}</groupId>
  <artifactId>${artifactId}</artifactId>
  <version>${version}</version>
  <packaging>war</packaging>
  <name>basicWebapp</name>
 
  <!-- Start elements of Oracle products and licenses -->
  <description>Oracle WebLogic Maven Archetype</description>
  <url>http://www.oracle.com/us/products/middleware/cloud-app-foundation/weblogic/suite/overview/index.html</url>
  <inceptionYear>1997</inceptionYear>
 
  <organization>
    <name>Oracle America, Inc.</name>
    <url>http://www.oracle.com</url>
  </organization>
  <licenses>
    <license>
      <name>
	The "Oracle Technology Network License Agreement - Oracle 
        Maven Repository" is the license for this Project Object 
	Model (POM) file and the associated software and documentation
	unless you have accepted or agreed to different license terms
	with Oracle when you obtained the right to use the POM file,
	software and documentation.  Please refer to the comments below
	for additional information.
      </name>
      <url>https://www.oracle.com/webapps/maven/register/license.html</url>
      <distribution>repo</distribution>
      <comments>
	Copyright (c) 2014, 2015, Oracle and/or its affiliates. All rights
	reserved.
 
	This Project Object Model (POM) file and the associated software and
	documentation are provided under a license agreement containing
	restrictions on use and disclosure and are protected by intellectual
	property laws. You acknowledge that your use of this POM file and the
	associated software and documentation is subject to the license terms
	that you accepted or agreed to when you obtained the right to use the
	POM file, software and documentation. In the event you have not
	accepted or agreed to any such license terms, you cannot use this
	POM file or the associated software and documentation without first
	accepting the Oracle Technology Network License Agreement - Oracle
	Maven Repository terms and conditions set forth at the above-listed
	URL.
 
	Except as expressly permitted in your license agreement or allowed
	by law, you may not:

	- use, copy, reproduce, translate, broadcast, modify, license,
	transmit, distribute, exhibit, perform, publish, or display any part,
	in any form, or by any means, the POM file or the associated software
	or documentation; and

	- deploy the POM file or the associated software or documentation to
	any Maven or other repositories except for your local Maven repository
	that is not accessible to any third party (other than contractors
	acting on your behalf if expressly permitted in your license
	agreement), including but not limited to the Maven Central Repository
	or any other public Maven repository.

	Reverse engineering, disassembly, or decompilation of this software,
	unless required by law for interoperability, is prohibited.  
 
	Additional Notices
 
	The information contained herein is subject to change without notice
	and is not warranted to be error-free. If you find any errors, please
	report them to us in writing.
 
	If this is software or related documentation that is delivered to the
	U.S. Government or anyone licensing it on behalf of the
	U.S. Government, the following notice is applicable:

	U.S. GOVERNMENT END USERS: Oracle programs, including any operating
	system, integrated software, any programs installed on the hardware,
	and/or documentation, delivered to U.S. Government end users are
	"commercial computer software" pursuant to the applicable Federal
	Acquisition Regulation and agency-specific supplemental regulations.
	As such, use, duplication, disclosure, modification, and adaptation
	of the programs, including any operating system, integrated software,
	any programs installed on the hardware, and/or documentation, shall be
	subject to the license terms and license restrictions applicable to the
	programs. No other rights are granted to the U.S. Government.

	Oracle and Java are registered trademarks of Oracle and/or its
	affiliates. Other names may be trademarks of their respective owners.

	This software or hardware and documentation may provide access to or
	information about content, products, and services from third parties.
	Oracle Corporation and its affiliates are not responsible for and
	expressly disclaim all warranties of any kind with respect to
	third-party content, products, and services unless otherwise set
	forth in an applicable agreement between you and Oracle. Oracle
	Corporation and its affiliates will not be responsible for any loss,
	costs, or damages incurred due to your access to or use of third-party
	content, products, or services, except as set forth in an applicable
	agreement between you and Oracle.
      </comments>
    </license>
  </licenses>
  <!-- End elements of Oracle products and licenses -->
  
  <parent>
    <groupId>com.oracle.weblogic.archetype</groupId>
    <artifactId>wls-common</artifactId>
    <version>14.1.2-0-0</version>
  </parent>
  <dependencies>
    <dependency>
      <groupId>javax</groupId>
      <artifactId>javaee-web-api</artifactId>
      <version>6.0</version>
      <scope>provided</scope>
    </dependency>
  </dependencies>
  <build>
    <finalName>basicWebapp</finalName>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>2.3.2</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>3.3.1</version>
        <configuration>
          <failOnMissingWebXml>false</failOnMissingWebXml>
        </configuration>
      </plugin>
      <plugin>
        <groupId>com.oracle.weblogic</groupId>
        <artifactId>weblogic-maven-plugin</artifactId>
        <version>14.1.2-0-0</version>
        <!--
        You can find and redefine the following variables in the parent pom file arccording to your environment.
  
        oracleMiddlewareHome
        oracleServerUrl
        oracleUsername
        oraclePassword
        oracleServerName
        -->
        <configuration>
          <middlewareHome>${oracleMiddlewareHome}</middlewareHome>
        </configuration>
        <executions>
          <!--Deploy the application to the server-->
          <execution>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>deploy</goal>
            </goals>
            <configuration>
              <!--The admin URL where the app is deployed. Here use the plugin's default value t3://localhost:7001-->
              <!--adminurl>${oracleServerUrl}</adminurl-->
              <user>${oracleUsername}</user>
              <password>${oraclePassword}</password>
              <!--The location of the file or directory to be deployed-->
              <source>${project.build.directory}/${project.build.finalName}.${project.packaging}</source>
              <!--The target servers where the application is deployed. Here use the plugin's default value AdminServer-->
              <!--targets>${oracleServerName}</targets-->
              <verbose>true</verbose>
              <name>${project.build.finalName}</name>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Hi @santhosh are those variables (${groupId} ${artifactId} ${version}) defined somewhere? I’m wondering if Syft has a bug where it spends forever trying to resolve some values in the pom.xml if it can’t, and that setting max-parent-recursive-depth=1 prevents this bug from occurring.

Thanks for getting back to me here! I’d love to try to make a specific bug to fix in Syft. Is the pom.xml that syft is hanging on in a publicly available JAR? If not maybe we can try to find or craft a pom.xml that causes the bug.

If you put the offending JAR in a directory by itself, does Syft hang scanning that directory?

You’re spot on @willmurphy – this must be a property resolution recursion issue.

I updated the post above to correctly format the XML contents; thanks for the report @santhosh!

I see this has the required groupId, artifactId, and version set referencing themselves(!):

  <groupId>${groupId}</groupId>
  <artifactId>${artifactId}</artifactId>
  <version>${version}</version>

Of course this should not result in a recursive hang or other error on the Syft side – I’ll get to work getting a fix in for it. But is this a valid way to specify a maven artifact?

Here’s a PR:

@kzantow Thanks for quick response. It is getting hung with any simple pom file which has variables.

I have few questions:

  1. What is the flag “max-parent-recursive-depth” used for? Will we be missing any package details if we reduce the depth from infinite to finite number?
  2. Why does syft hanging when there are variables? If the variable value is not available, it should just say unable to resolve and proceed further right?
  3. Why does it work after providing finite value? Still, there are variables but with finite depth, it worked?

Could you please help me understand the reason for hung?

Regarding sample PR, it is not publicly available. Scan is getting stuck even with simple pom file

https://pastebin.com/MPph4Zvd

Hi @santhosh,

  1. What is the flag “max-parent-recursive-depth” used for? Will we be missing any package details if we reduce the depth from infinite to finite number?

max-parent-recursive-depth is a setting which causes Syft to stop attempting to resolve a variable after checking that number of parents. So 1 would try only one parent level but not another. Yes, you could miss details if this is set too low, which is why by default it is set to unlimited.

  1. Why does syft hanging when there are variables?

Syft resolves variables based on similar logic to what Maven uses to resolve them, which often includes using values defined in parent POMs – if the value isn’t available directly in the POM.

If the variable value is not available, it should just say unable to resolve and proceed further right?

If the value isn’t available, it should continue on, yes. There were a couple problems working together which caused Syft to get this POM incorrectly referencing itself as its own parent, creating a cycle, and this particular code path was not correctly checking for cycles, which is why there was a hang. Apologies for this – we really tried to make sure that all the recursive maven resolution logic had appropriate checks to prevent infinite recursion like this but clearly missed a case.

@kzantow by any chance, do you know how can I install non-latest version of syft in my mac m3 chip?

I tried brew install syft@v1.1.1 but its not working when I specify version.

I don’t know about brew, none of the incantations seemed to work for me.

However, you can use the install.sh method. For example – to get the syft binary into the current dir (the dot . in the script below), at version v1.11.1, you could use:

curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b . v1.11.1

We can get a patch release out fairly soon with the fix for you, however – keep an eye out!

1 Like