Decide if we want to index products of package dependencies

[ahoppen]

We currently index all products of a package’s dependencies. This matches the fact that you can build all all targets referenced by a dependency’s product on the command line (eg. you can run swift build --product swift-driver from within SourceKit-LSP. Most users probably never use this fact though and we already don’t index all code in package dependencies (eg. test targets of dependencies aren’t indexed). Based on initial measurements, we could save about another minute (relative to 2:45 min) on the total indexing time of SourceKit-LSP if we didn’t index products of dependencies.

If we want to go this way, we’d likely need to check if a product is in graph.reachableProducts here: https://github.com/swiftlang/swift-package-manager/blob/3a4ba00c1fbdf119512f2293a51d79521e664518/Sources/Build/BuildPlan/BuildPlan.swift#L980-L984

Stub for a test case

func testFoo() async throws {
  let project = try await MultiFileTestProject(
    files: [
      // MyLibrary
      "MyDependency/Sources/MyDependency/MyDependency.swift": """
      public func makeHello() -> String { "" }
      """,
      "MyDependency/Sources/swift-help/swift-help.swift": "",
      "MyDependency/Package.swift": """
      // swift-tools-version: 5.7

      import PackageDescription

      let package = Package(
        name: "MyDependency",
        products: [
          .library(name: "MyDependency", targets: ["MyDependency"]),
          .executable(name: "swift-help", targets: ["swift-help"]),
        ],
        targets: [
          .target(name: "MyDependency"),
          .executableTarget(name: "swift-help")
        ]
      )
      """,

      // MyPackage
      "MyPackage/Sources/MyPackage/MyPackage.swift": """
      import MyDependency

      func test() {
        let x: Void = makeHello()
      }
      """,
      "MyPackage/Package.swift": """
      // swift-tools-version: 5.7

      import PackageDescription

      let package = Package(
        name: "MyPackage",
        dependencies: [
          .package(path: "../MyDependency"),
        ],
        targets: [.target(name: "MyPackage", dependencies: [.product(name: "MyDependency", package: "MyDependency")])]
      )
      """,
    ],
    workspaces: { scratchDirectory in
      return [WorkspaceFolder(uri: DocumentURI(scratchDirectory.appending(component: "MyPackage")))]
    },
    enableBackgroundIndexing: true
  )
  try await project.testClient.send(SynchronizeRequest(index: true))
}

[DPrakashhh]

Hi, I have been looking into this and have a working implementation.

You mentioned checking graph.reachableProducts in BuildPlan.swift. In my current approach, I've implemented the filtering in SwiftPMBuildServer.swift within the traverseModules loop by pre-computing a set of reachable targets from the modulesGraph.

This correctly filters out unreachable dependency products (like the swift-help example) while keeping the root package and required products indexed. Before I submit the PR, would you prefer the logic to remain in the LSP server, or should I move the filtering deeper into the SwiftPM BuildPlan logic as originally suggested?

[ahoppen]

I think there’s a general question about whether we want to do this or not. I personally think we should but don’t want to make this call on my own. I’d like to hear @hamishknight and @rintaro’s opinion on it first.

Let’s see how your implementation purely in SourceKit-LSP looks like. If it’s reasonable, I think it would be nice to not infiltrate SwiftPM with this logic. We can always adjust after seeing how nice the code looks.

[DPrakashhh]

Sure, I will work on the implementation, after index-dump and Hide 'Convert to computed property' PR's , it's getting confusing now thinking about multiple issues at the same time :D, so i will focus on them for now sir.

[rintaro]

Could you explain the consequences of this? "quick open" won't find anything declared in non-indexed code, "find references" won't find usages in non-indexed code, anything else?

If that's it, I don't see any reason not to do this. Actually, I feel it's harmful to find anything unrelated to current package.

[DPrakashhh]

You’re exactly right—those are the trade-offs I was seeing too.

I actually agree that it's often more distracting than helpful to see internal tools from a dependency in search results. Since I’m still learning the ins and outs of the indexer, I wanted to be extra careful: I have implemented this to always index the "root package" 100%, even if a target isn't reachable yet. That way, we get the performance boost without risking any of the user's own code being hidden.

Does this approach seem like the right direction to you? I would love to hear what you think!

[ahoppen]

One other potential downside is that you won’t get semantic functionality if you jump into a file of one of these non-reachable targets in one of your dependencies (at least I think that’s how it will work, we might also then open the dependency package a separate SourceKit-LSP workspace and index it from scratch). This behavior will match what you get when you open a test file in one of your dependencies.

[DPrakashhh]

That's a very helpful point. I hadn't fully considered the 'dead end' when navigating into a dependency's internal files.

However, since this behavior will match how test targets in dependencies are already handled, I think it sets a consistent precedent. Most users likely expect full semantic support for the libraries they are actually using, but might find the ~35% indexing speedup a more significant daily benefit than having 'Jump to Definition' inside a dependency's benchmarking tool or helper executable.

By ensuring the root package is always 100% indexed, we protect the user's primary workspace. If this trade-off seems acceptable to the team, I’m happy to address any specific feedback on the implementation in #2444.

[rintaro]

Sorry I didn't make it clear sooner, but I'm all for this change.