Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
7af2fc9
feat: implement DESTROY, weaken/isweak/unweaken with refCount tracking
fglock Apr 8, 2026
c8e8599
fix: weaken/refCount improvements — 178/196 sandbox tests passing
fglock Apr 8, 2026
a7375b6
fix: scoped MortalList flush + re-bless refCount -- 186/196 sandbox t…
fglock Apr 8, 2026
65b2ae1
fix: AUTOLOAD DESTROY, cascading destruction, container scope cleanup…
fglock Apr 8, 2026
bc9d357
fix: splice refCount tracking for blessed elements -- 194/196 sandbox…
fglock Apr 8, 2026
ad9857b
fix: closure capture tracking for DESTROY -- 196/196 sandbox tests
fglock Apr 8, 2026
2aaa670
docs: warn against gh pr create --body with backticks in AGENTS.md
fglock Apr 8, 2026
f178247
test: move destroy/weaken sandbox tests to src/test/resources/unit/re…
fglock Apr 8, 2026
e911a14
fix: prevent premature weak ref clearing for non-DESTROY objects
fglock Apr 8, 2026
08798bc
fix: caller() without args returns 3 elements, fix local @_ in JVM ba…
fglock Apr 8, 2026
3d7ec7a
docs: update moo_support.md with Phase 40-41 results (68/71 tests)
fglock Apr 8, 2026
1b5ba98
Add POSIX::_do_exit for demolish-global_destruction.t
fglock Apr 8, 2026
c677a14
docs: update destroy_weaken_plan.md to v5.7 with findings
fglock Apr 8, 2026
c493110
feat: CPAN distroprefs for Moo installation (jcpan -i Moo)
fglock Apr 8, 2026
0cbc81a
feat: implement lazy refCount tracking for deterministic weak-ref cle…
fglock Apr 8, 2026
5bc3950
fix: refCount tracking for container stores, re-bless, and closure ca…
fglock Apr 9, 2026
0b21903
fix: release eval STRING captures after execution to unblock weak ref…
fglock Apr 9, 2026
54d912f
fix: track refCounts for elements in anonymous array/hash construction
fglock Apr 9, 2026
124df83
fix: prevent premature weak ref clearing for unblessed birth-tracked …
fglock Apr 9, 2026
851b16c
fix: transition unblessed weakened objects to WEAKLY_TRACKED to preve…
fglock Apr 9, 2026
63e7a81
fix: destroy anonymous unblessed objects when last strong ref is weak…
fglock Apr 9, 2026
4200e5c
docs: update Moo test plan with Category B fix results
fglock Apr 9, 2026
075927c
fix: emulate optree reaping to clear weak refs when subroutine is rep…
fglock Apr 9, 2026
53de468
docs: update Moo plan - 841/841 subtests passing (100%)
fglock Apr 9, 2026
8c0b787
docs: add weaken/DESTROY architecture document
fglock Apr 9, 2026
25d2e9d
fix: built-in functions no longer shadowed by same-name sub definitions
fglock Apr 9, 2026
d894670
docs: add benchmark results to weaken/DESTROY architecture doc
fglock Apr 9, 2026
a82abcc
fix: DESTROY warning format + architecture doc update
fglock Apr 9, 2026
13294de
docs: add Strategy A experimental results and future strategy analysis
fglock Apr 9, 2026
9de4f5b
fix: prevent premature weak ref clearing for untracked objects (qr-72…
fglock Apr 9, 2026
d99aa57
Move post-merge action items from architecture doc to plan doc
fglock Apr 9, 2026
4926a48
Fix architecture doc to match actual callDestroy() implementation
fglock Apr 9, 2026
aea63c6
fix: refcount leaks in list destructuring, container stores, and weak…
fglock Apr 9, 2026
a492681
fix: force-clear weak refs on explicit undef of unblessed objects
fglock Apr 9, 2026
c68c274
docs: update design doc with force-clear findings and v5.8 progress
fglock Apr 9, 2026
9af7cb2
docs: document approaches tried and reverted for WEAKLY_TRACKED
fglock Apr 9, 2026
043271f
fix: skip weak ref clearing for CODE objects (fixes 46 Moo test failu…
fglock Apr 9, 2026
74966d6
fix: fire DESTROY on untie via refcounting (matches Perl 5)
fglock Apr 9, 2026
2c38106
fix: refCount leak in list destructuring and scope exit cleanup
fglock Apr 9, 2026
8e5acf6
fix: always decrement refCount at scope exit for captured variables
fglock Apr 10, 2026
b01bd45
fix: release eval BLOCK captures eagerly to prevent weak ref leaks
fglock Apr 10, 2026
31a80b8
docs: update destroy_weaken_plan.md with eval BLOCK capture release f…
fglock Apr 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
477 changes: 477 additions & 0 deletions .cognition/skills/debug-exiftool/SKILL.md

Large diffs are not rendered by default.

424 changes: 424 additions & 0 deletions .cognition/skills/debug-perlonjava/SKILL.md

Large diffs are not rendered by default.

187 changes: 187 additions & 0 deletions .cognition/skills/debug-windows-ci/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
# Debug PerlOnJava Windows CI Failures

## ⚠️⚠️⚠️ CRITICAL: NEVER USE `git stash` ⚠️⚠️⚠️

**DANGER: Changes are SILENTLY LOST when using git stash/stash pop!**

- NEVER use `git stash` to temporarily revert changes
- INSTEAD: Commit to a WIP branch or use `git diff > backup.patch`
- This warning exists because completed work was lost during debugging

## Overview

This skill helps debug test failures that occur specifically in the Windows CI/CD environment but pass locally on macOS/Linux.

## When to Use

- Tests pass locally on macOS/Linux but fail on Windows CI
- Windows-specific path handling issues
- Shell command differences between platforms
- File I/O issues on Windows

## CI/CD Structure

### GitHub Actions Workflow

The CI runs on `windows-latest` using:
- Java 21 (Temurin)
- Gradle for build
- Maven for tests (`make ci` runs `mvn clean test`)

### Viewing CI Logs

```bash
# List recent CI runs
gh run list --branch <branch-name> --limit 5

# View failed test logs
gh run view <run-id> --log-failed

# Filter for specific errors
gh run view <run-id> --log-failed 2>&1 | grep -E "FAILURE|error|not ok"

# Get test count summary
gh run view <run-id> --log-failed 2>&1 | grep "Tests run:"
```

## Common Windows CI Issues

### 1. Cwd/getcwd Issues

**Symptom**: "Cannot chdir back to : 2" or "Undefined subroutine &Cwd::cwd called"

**Root Cause**: The Perl `Cwd.pm` uses shell backticks (`` `cd` ``) on Windows which doesn't work in PerlOnJava.

**Solution**: PerlOnJava provides `Internals::getcwd` which uses Java's `System.getProperty("user.dir")`. The Cwd.pm has been modified to use this when available.

**Key Files**:
- `src/main/perl/lib/Cwd.pm` - Perl module with platform-specific fallbacks
- `src/main/java/org/perlonjava/runtime/perlmodule/Internals.java` - Java implementation of getcwd

### 2. Temp File Creation Issues

**Symptom**: "Cannot open/create <filename>: open failed"

**Root Cause**:
- Windows uses different path separators (`\` vs `/`)
- Temp directory permissions may differ
- File locking behavior differs on Windows

**Debugging**:
```bash
# Check temp path in error message
gh run view <run-id> --log-failed 2>&1 | grep "open failed"
```

### 3. $^O Detection

PerlOnJava sets `$^O` based on the Java `os.name` property:
- Windows: `MSWin32`
- macOS: `darwin`
- Linux: `linux`

**Key File**: `src/main/java/org/perlonjava/runtime/runtimetypes/SystemUtils.java`

### 4. Shell Command Differences

Windows CI may fail when Perl code uses:
- Backticks with Unix commands
- `system()` calls assuming Unix shell
- Path separators in shell commands

## Debugging Workflow

### Step 1: Identify the Failing Test

```bash
# Get list of failing tests
gh run view <run-id> --log-failed 2>&1 | grep "testUnitTests.*FAILURE"
```

### Step 2: Map Test Number to File

```bash
# List tests in order (tests are numbered alphabetically)
ls -1 src/test/resources/unit/*.t | sort | nl | grep "<number>"
```

### Step 3: Analyze the Error

```bash
# Get full context around error
gh run view <run-id> --log-failed 2>&1 | grep -A10 "unit\\<test>.t"
```

### Step 4: Check if Pre-existing

```bash
# Compare with master branch CI
gh run list --branch master --limit 3
gh run view <master-run-id> --log-failed
```

## Platform-Specific Code Patterns

### Checking for Windows in Perl

```perl
if ($^O eq 'MSWin32') {
# Windows-specific code
}
```

### Checking for Windows in Java

```java
if (SystemUtils.osIsWindows()) {
// Windows-specific code
}
```

### Safe Cross-Platform getcwd

```perl
# In Cwd.pm, use Internals::getcwd if available
if (eval { Internals::getcwd(); 1 }) {
*getcwd = \&Internals::getcwd;
}
```

## Test File Locations

- Unit tests: `src/test/resources/unit/*.t`
- Perl5 test suite: `perl5_t/t/`
- Java tests: `src/test/java/org/perlonjava/`

## Related Files

- `.github/workflows/gradle.yml` - CI workflow definition
- `Makefile` - Build targets including `ci`
- `src/main/java/org/perlonjava/runtime/perlmodule/Cwd.java` - Java Cwd stub
- `src/main/perl/lib/Cwd.pm` - Perl Cwd implementation

## Troubleshooting Checklist

1. [ ] Is the failure Windows-specific? (Check if macOS/Linux CI passes)
2. [ ] Is it a new regression or pre-existing? (Compare with master)
3. [ ] Does it involve file paths or shell commands?
4. [ ] Does it use Cwd or directory operations?
5. [ ] Is `$^O` being checked correctly?
6. [ ] Are there any `defined &Subroutine` checks that might behave differently?

## Adding Debug Output

To debug CI issues, you can temporarily add print statements to Perl modules:

```perl
# Add to Cwd.pm to debug
warn "DEBUG: \$^O = $^O";
warn "DEBUG: Internals::getcwd available: " . (eval { Internals::getcwd(); 1 } ? "yes" : "no");
```

Then check CI logs:
```bash
gh run view <run-id> --log-failed 2>&1 | grep "DEBUG:"
```

Remember to remove debug output before final commit.
207 changes: 207 additions & 0 deletions .cognition/skills/debugger/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
# Perl Debugger Implementation Skill

## ⚠️⚠️⚠️ CRITICAL: NEVER USE `git stash` ⚠️⚠️⚠️

**DANGER: Changes are SILENTLY LOST when using git stash/stash pop!**

- NEVER use `git stash` to temporarily revert changes
- INSTEAD: Commit to a WIP branch or use `git diff > backup.patch`
- This warning exists because completed work was lost during debugging

## Overview

Continue implementing the Perl debugger (`-d` flag) for PerlOnJava. The debugger uses DEBUG opcodes injected at statement boundaries in the bytecode interpreter.

## Git Workflow

**IMPORTANT: Never push directly to master. Always use feature branches and PRs.**

**IMPORTANT: Always commit or stash changes BEFORE switching branches.** If `git stash pop` has conflicts, uncommitted changes may be lost.

```bash
git checkout -b feature/debugger-improvement
# ... make changes ...
git push origin feature/debugger-improvement
gh pr create --title "Debugger: description" --body "Details"
```

## Key Documentation

### Design Document
- **Location**: `dev/design/perl_debugger.md`
- Contains implementation phases, architecture diagrams, and code examples

### Perl Debugger Documentation (reference)
- `perldoc perldebug` - User documentation for Perl debugger
- `perldoc perldebguts` - Internal implementation details (key reference!)
- `perldoc perldebtut` - Tutorial
- `perl5/lib/perl5db.pl` - The standard Perl debugger (~10,000 lines)

## Current Implementation Status

**Branch**: `implement-perl-debugger`

### Completed (Phase 1 + partial Phase 2)
- DEBUG opcode (376) in `Opcodes.java`
- `-d` flag in `ArgumentParser.java` sets `debugMode=true`, forces interpreter
- `BytecodeCompiler` emits DEBUG at statement boundaries when `debugMode=true`
- `BytecodeInterpreter` handles DEBUG opcode, calls `DebugHooks.debug()`
- `DebugState.java` - global debug flags, breakpoints, source storage
- `DebugHooks.java` - command loop with n/s/c/q/l/b/B/L/h commands
- Source line extraction from tokens (`ErrorMessageUtil.extractSourceLines()`)
- `l` command shows source with `==>` current line marker
- Compile-time statements (`use`/`no`) correctly skipped via `compileTimeOnly` annotation
- Infrastructure nodes in BEGIN blocks skipped via `skipDebug` annotation

### Working Commands
| Command | Description |
|---------|-------------|
| `n` | Next (step over) |
| `s` | Step into (shows subroutine name, e.g., `main::foo(file:line)`) |
| `r` | Return (step out of current subroutine) |
| `c [line]` | Continue (optionally to line) |
| `q` | Quit |
| `l [range]` | List source (`l 10-20` or `l 15`) |
| `.` | Show current line |
| `b [line]` | Set breakpoint |
| `B [line]` | Delete breakpoint (`B *` = all) |
| `L` | List breakpoints |
| `T` | Stack trace |
| `p expr` | Print expression (supports lexical variables) |
| `x expr` | Dump expression with Data::Dumper (supports lexical variables) |
| `h` | Help |

## Comparison with System Perl Debugger

Tested side-by-side with `perl -d`:

| Feature | jperl | System perl | Status |
|---------|-------|-------------|--------|
| Start line | First runtime stmt | First runtime stmt | Match |
| `n` (next) | Works | Works | Match |
| `s` (step) | Works | Works | Match |
| `c` (continue) | Works | Works | Match |
| `b` (breakpoint) | Works, confirms | Works, silent | OK |
| `L` (list bp) | Simple list | Shows code + condition | Different |
| `l` (list) | Shows context around line | Shows current line only | Different |
| `q` (quit) | Works | Works | Match |
| Package prefix | Missing | Shows `main::` | TODO |
| Prompt counter | `DB<0>` (0-indexed) | `DB<1>` (1-indexed) | TODO |
| Loading message | None | Shows perl5db.pl version | OK (intentional) |

### Known Differences to Address
1. ~~**Package prefix**: Add `main::` (or current package) to location display~~ **DONE**
2. ~~**Prompt counter**: Change to 1-indexed (`DB<1>`) to match Perl~~ **DONE**
3. **`l` command**: Perl shows current line, subsequent `l` shows next 10 lines

## Source Files

| File | Purpose |
|------|---------|
| `src/main/java/org/perlonjava/runtime/debugger/DebugState.java` | Global flags, breakpoints, source storage |
| `src/main/java/org/perlonjava/runtime/debugger/DebugHooks.java` | Debug hook called by DEBUG opcode, command loop |
| `src/main/java/org/perlonjava/backend/bytecode/Opcodes.java` | DEBUG = 376 |
| `src/main/java/org/perlonjava/backend/bytecode/BytecodeCompiler.java` | Emits DEBUG opcodes, checks `skipDebug` |
| `src/main/java/org/perlonjava/backend/bytecode/BytecodeInterpreter.java` | Handles DEBUG opcode |
| `src/main/java/org/perlonjava/app/cli/ArgumentParser.java` | `-d` flag handling |
| `src/main/java/org/perlonjava/frontend/parser/StatementParser.java` | Marks `use`/`no` as `compileTimeOnly` |
| `src/main/java/org/perlonjava/frontend/parser/SpecialBlockParser.java` | Marks BEGIN infrastructure as `skipDebug` |
| `src/main/java/org/perlonjava/runtime/runtimetypes/ErrorMessageUtil.java` | `extractSourceLines()` for source display |

## Next Steps (from design doc)

### Phase 2: Source Line Support (mostly done)
- [x] Store source lines during parsing
- [x] Skip compile-time statements (use/no)
- [x] Display subroutine names when stepping (e.g., `main::foo(file:line)`)
- [ ] Track breakable lines (statements vs comments)
- [ ] Implement `@{"_<$filename"}` magical array
- [ ] Implement `%{"_<$filename"}` for breakpoint storage

### Phase 3: Debug Variables (partially done)
- [x] `$DB::single`, `$DB::trace`, `$DB::signal` synced from Java
- [x] `$DB::filename`, `$DB::line` set by DEBUG opcode
- [x] `@DB::args` support in `caller()`
- [x] `%DB::sub` for subroutine location tracking
- [ ] Make debug variables fully tied (Perl can modify them)

### Phase 4: Perl Expression Evaluation (DONE)
- [x] `p expr` - print expression value
- [x] `x expr` - dump expression (Data::Dumper style)
- [x] Lexical variable access in debugger expressions
- [x] Registry deduplication to minimize memory usage

### Phase 5: perl5db.pl Compatibility
- [ ] Inject `BEGIN { require 'perl5db.pl' }` when `-d` used
- [ ] `DB::sub()` routing for subroutine tracing
- [ ] Test with actual perl5db.pl

## Tips for Development

**ALWAYS use `make` commands. NEVER use raw mvn/gradlew commands.**

| Command | What it does |
|---------|--------------|
| `make` | Build + run all unit tests (use before committing) |
| `make dev` | Build only, skip tests (for quick iteration during debugging) |

### Testing the debugger
```bash
make dev # Quick build after changes (no tests)

# Test basic stepping
echo 'n
n
q' | ./jperl -d /tmp/test.pl

# Test source listing
echo 'l
l 1-10
q' | ./jperl -d -e 'print 1; print 2; print 3;'

# Test breakpoints
echo 'b 3
c
q' | ./jperl -d /tmp/test.pl

# Compare with system perl
perl -d /tmp/test.pl
```

### Interactive testing
The debugger can be tested interactively - send commands and observe responses.

### Key design principles
1. **All debugger logic in DebugHooks** - interpreter loop stays clean
2. **Zero overhead when not debugging** - no DEBUG opcodes emitted
3. **Breakpoints via Set<String>** - O(1) lookup of "file:line"
4. **Source from tokens** - `ErrorMessageUtil.extractSourceLines()` rebuilds source
5. **Skip internal nodes** - `compileTimeOnly` and `skipDebug` annotations

### Adding new commands
1. Add case in `DebugHooks.executeCommand()`
2. Create `handleXxx()` method
3. Return `true` to resume execution, `false` to stay in command loop
4. Update `handleHelp()` with new command

### Adding debug variables
To expose `$DB::single` etc. to Perl code:
1. Create tied variable class that reads/writes `DebugState` fields
2. Register in GlobalVariable initialization
3. See `GlobalVariable.java` for examples of special variables

### Step-over implementation
Already working via `DebugState.stepOverDepth`:
- `n` sets `stepOverDepth = callDepth`
- DEBUG skips when `callDepth > stepOverDepth`
- Need to call `DebugHooks.enterSubroutine()`/`exitSubroutine()` on sub entry/exit

### Annotations for skipping DEBUG opcodes
- `compileTimeOnly` - skips entire statement compilation (for `use`/`no` results)
- `skipDebug` - skips only DEBUG opcode emission (for infrastructure nodes)

### Common issues
- **Source not showing**: Check `DebugState.sourceLines` is populated
- **Breakpoint not hitting**: Verify line is breakable (has DEBUG opcode)
- **Step-over not working**: Ensure `callDepth` tracking is correct
- **Duplicate lines**: Check for missing `skipDebug` on internal nodes
Loading