D 2025-01-10T06:05:44.753
L Documentation
N text/x-markdown
P 688196daf451f1ea42b1a03254a5a74b9959ee84
U mistachkin
W 16217
Preliminary documentation for <b>v1</b> of the Package Client is available <b>[/doc/trunk/doc/v1.html|here]</b>.

---

Below is a set of **four man-page–style documents** covering each core script in the Eagle “package client toolset.” They follow a **Tcl SDK–style** documentation format, with sections like **Name**, **Synopsis**, **Description**, **Commands**, **Options**, **Examples**, **Bugs/Limitations**, **See Also**, and **Keywords**. Each script—**`pkgr.eagle`**, **`pkgd.eagle`**, **`pkgu.eagle`**, and **`common.tcl`**—is documented separately.

---

# 1. **pkgr.eagle** — Package Repository Client

## NAME
**pkgr.eagle** — Eagle Package Repository Client

## SYNOPSIS
```tcl
source pkgr.eagle
```
*(Assumes you have placed `pkgr.eagle` in your script library or added its directory to `::auto_path`. This script can be loaded automatically by `package require Eagle.Package.Repository`.)*

## DESCRIPTION
The **pkgr.eagle** script provides a client interface to the remote Eagle **package repository**. It can:

1. **Hook** into Tcl’s or Eagle’s `[package unknown]` mechanism, so that missing packages are fetched automatically from a remote server.
2. **Download** package metadata, scripts, and signatures using either `[uri download]` (Eagle) or `[::http::geturl]` (Tcl).
3. **Verify** package scripts and certificates (OpenPGP or Harpy).
4. **Offer** a fallback for anonymous vs. API-key–based access.

In short, it helps your application load packages on demand from a central repository, with security checks and minimal local overhead.

### Primary Features
- **Inline** hooking of `[package unknown]`: If a package is not found locally, `pkgr.eagle` queries the remote server, downloads the script, verifies signatures, and registers it in `[package ifneeded ...]`.
- **OpenPGP verification**: Ensures scripts are signed with GPG/gnupg (for example, `gpg2`).
- **Harpy certificates**: Supports advanced script signing with Harpy (for `.eagle` scripts).
- **Namespace**: All functionality is in `::PackageRepository`.

## COMMANDS
Below is a summary of the primary commands provided (or influenced) by **pkgr.eagle**. Most live under the `::PackageRepository` namespace.

1. **`setupRepositoryServerVars force`**  
   Initializes or resets server URN variables for package lookups (e.g. `lookupUrn`, `submitUrn`, etc.).  

2. **`setupRepositoryUriVars force`**  
   Initializes or resets URI variables (e.g. `baseUri`, `lookupUri`, etc.).

3. **`setupPackageUnknownVars force`**  
   Sets up defaults (API keys, hooking preferences, verbosity) for the `[package unknown]` mechanism.

4. **`setupPackageUnknownHandler`**  
   Optionally installs the custom `[package unknown]` handler that automatically fetches missing packages from the repository.

5. **`packageUnknownHandler package ?version?`**  
   The actual `[package unknown]` hook. Called by Tcl/Eagle if a requested package is not found locally. In turn, it attempts `getPackageFromRepository`.

6. **`getPackageFromRepository package version caller`**  
   The central routine that requests metadata from the remote server, verifies the script’s signature, and registers the package.

7. **`processLookupMetadata metadataVarName`**  
   Given extracted metadata (script, certificate), verifies and loads them, respecting either “Tcl” or “Eagle” language.

8. **`extractAndVerifyLookupMetadata result varName caller`**  
   Parses the raw repository response, checks name/patchLevel/language/script/certificate, and stores them into `varName`.

9. **`probeForOpenPgpInstallation`, `openPgpMustBeInstalled`**  
   Check for GPG/gnupg presence and appropriate version.

10. **`verifyOpenPgpSignature fileName`**  
    Returns a boolean indicating if the specified OpenPGP signature file is valid.

... and many more utility procedures like `isEagle`, `eagleMustBeReady`, `makeEagleReady`, `stringIsList`, `formatResult`, etc.

## OPTIONS
- **`autoHook`**: If true, automatically hooks `[package unknown]`.  
- **`verboseUnknownResult`**: If true, logs debug info each time `[package unknown]` is invoked.  
- **`forceSecureUri`**: If true, upgrades `http://` to `https://` when TLS is available.  
- **`mustHaveTls`**: If true, raises errors if `tls` cannot be loaded.  
- **`strictUnknownLanguage`**: If true, disallows loading a package if its declared language doesn’t match the current interpreter.

## EXAMPLES
```tcl
# Basic usage in Eagle:
package require Eagle.Package.Repository

# Automatic hooking:
::PackageRepository::setupPackageUnknownVars false
::PackageRepository::setupPackageUnknownHandler

# Now [package require SomeRemotePackage] will fetch from the remote repo.
```

## BUGS AND LIMITATIONS
1. **Strict security**: By default, unknown scripts must have valid signatures (OpenPGP or Harpy). This can fail if `gpg` is not installed or if the signature is missing/invalid.
2. **Non-automatic**: While it can be automatic, you must ensure your environment is configured properly (`auto_path`, hooking, etc.).
3. **Limited OS checks**: Some logic (like IPv6 vs. IPv4 forcing) is partial and may not handle all edge cases.

## SEE ALSO
- **`pkgd.eagle`** — Package Downloader
- **`pkgu.eagle`** — Package Uploader
- **`common.tcl`** — Shared HTTP and file utilities
- **[Fossil SCM](https://fossil-scm.org/)** if using the “file repository” approach

## KEYWORDS
Packages, Repository, Eagle, Tcl, Download, `[package unknown]`, OpenPGP, Harpy, Signing

---

# 2. **pkgd.eagle** — Package Downloader Client

## NAME
**pkgd.eagle** — Eagle Package Downloader Client

## SYNOPSIS
```tcl
source pkgd.eagle
```
*(Often invoked via `package require Eagle.Package.Downloader`.)*

## DESCRIPTION
**pkgd.eagle** focuses on **downloading** package files from a remote server, optionally persisting them, verifying signatures, and bridging them into **Tcl** or **Eagle** interpreters. It also handles some multi-language awareness: e.g., an Eagle script can fetch `.tcl` packages and vice versa (with the help of *Garuda* on Tcl or *tclMustBeReady* in Eagle).

### Key Capabilities
1. **Download**: Using `getPackageFile` or `downloadFiles` with either standard `[http]` or `[uri download]`.
2. **Signature checks**: OpenPGP or Harpy.
3. **Persistent vs. Temporary**: Files can be placed in a permanent directory or a temporary one.  
4. **Auto-path** modifications**: If `-useAutoPath` is true, newly downloaded directories get appended to the auto-path.

## COMMANDS
All commands reside in `::PackageDownloader`:

1. **`downloadFiles language version platform fileNames options`**  
   Downloads a list of files from the package server.  
   - `-persistent`: store them permanently.  
   - `-usePgp`: enable signature verification.  
   - `-useAutoPath`: add directories containing the downloaded files to `::auto_path`.

2. **`downloadOneFile language version platform fileName localFileName usePgp`**  
   A low-level routine for fetching a single file and verifying if `usePgp` is set.

3. **`resetCookieAndLogin userName password`** / **`logoutAndResetCookie`**  
   Handles session cookies for private or public logins to the remote file server.

4. **`probeForOpenPgpInstallation`**, **`openPgpMustBeInstalled`**  
   Similar to pkgr.eagle, ensures GPG is present.

5. **`maybeAddToAutoPath language directory`**  
   Appends a directory to the auto-path for the specified language (Tcl or Eagle) if it’s not already present.

6. **`getLookupData`, `extractAndVerifyLookupMetadata`,** etc.  
   Utility procedures for internal or advanced usage.

## OPTIONS
- **`-persistent`**: Boolean. If true, downloaded files go into the “persistentRootDirectory.”  
- **`-useAutoPath`**: Boolean. If true, automatically add the downloaded directory to `[auto_path]`.  
- **`-usePgp`**: Boolean. If true, require an OpenPGP signature for each file.  
- **`-allowUpdate`**: Boolean. If true, overwrites existing package files.

## EXAMPLES
```tcl
package require Eagle.Package.Downloader

# Download a single file in Eagle:
::PackageDownloader::downloadOneFile eagle 1.0 neutral \
    mypkg.eagle /tmp/mypkg.eagle true

# Download multiple, persist, and auto-path them:
::PackageDownloader::downloadFiles tcl 8.6 "" {pkgIndex.tcl README.txt} {
    -persistent true
    -useAutoPath true
    -usePgp true
}
```

## BUGS AND LIMITATIONS
1. **Cross-language bridging**: Relies on Eagle’s `[tclMustBeReady]` or Tcl’s `[eagleMustBeReady]`.  
2. **File collisions**: If `-allowUpdate` is `false`, existing files cause errors.  
3. **Security**: Temp directories must be local and not shared with untrusted processes.

## SEE ALSO
- **`pkgr.eagle`** — Repository Client (handles `[package unknown]`)  
- **`pkgu.eagle`** — Uploader  
- **`common.tcl`** — Shared utility code  

## KEYWORDS
Downloader, Eagle, Tcl, Packages, HTTP, GPG, auto-path

---

# 3. **pkgu.eagle** — Package Uploader Client

## NAME
**pkgu.eagle** — Eagle Package Uploader Client

## SYNOPSIS
```tcl
source pkgu.eagle
```
*(Often invoked via `package require Eagle.Package.Uploader`.)*

## DESCRIPTION
**pkgu.eagle** manages **uploading** (staging and committing) new packages into a Fossil-based “package file server” and then **submitting** metadata to the remote repository. It expects:

- **Local** Fossil checkout of the “package file server” repository.
- **Proper** signings (OpenPGP and optionally Harpy for `.eagle` scripts).
- **Connectivity** to the same server used by `pkgr.eagle` and `pkgd.eagle`.

### Key Capabilities
1. **Stage new package files** in the local Fossil checkout (`stagePackageFiles`).  
2. **Commit** them with a custom branch name (`commitPackageFiles`).  
3. **Generate** a repository script that references those newly committed files.  
4. **Submit** the script and signature to the remote server (`submitPackageMetadata`).

## COMMANDS
Namespace: `::PackageUploader`

1. **`stagePackageFiles language version platform fileNames`**  
   - Verifies no local pending changes, correct project/branch.  
   - Copies the given files into `packages/<lang>/<version>/<platform>` in the Fossil checkout.  
   - Runs `fossil add`, signs them with OpenPGP/Harpy.

2. **`commitPackageFiles package patchLevel language version varName`**  
   - Fossil commit with a message referencing the package name, patch level, language, and local checkout ID.  
   - If success, stores the new checkin ID in `varName`.

3. **`createRepositoryScript serverId versionId language version platform fileNames options`**  
   - Produces an inline Eagle script that can be used by the repository client to download these newly staged files.

4. **`submitPackageMetadata apiKey package patchLevel language script certificate`**  
   - POSTs a multipart form to the repository’s “submit” endpoint, telling it about the newly available package version.

5. **`getCheckoutDirectory`, `verifyCheckoutDirectory`**  
   - Ensures the local Fossil repo is present and valid.

6. **`fossilMustBeInstalled`, `verifyThereAreNoChanges`, `verifyThisIsTheCorrectProject`, `verifyThisIsTheCorrectBranch`**  
   - Housekeeping checks for Fossil usage.

## OPTIONS
- **`projectCode`**: The known project code for the Fossil repo. If mismatched, `verifyThisIsTheCorrectProject` fails.
- **`fossilFileNameOnly`**: Name of the Fossil CLI executable, default `fossil.exe` or `fossil`.
- **`scriptDirectory`**: Directory containing the package client tools; used to detect the local checkout.

## EXAMPLES
```tcl
package require Eagle.Package.Uploader

# Stage & commit a new package version:
set files [list /path/to/MyPkg.eagle /path/to/pkgIndex.eagle]
::PackageUploader::stagePackageFiles eagle 1.0 win64-x64 $files
::PackageUploader::commitPackageFiles MyPkg 1.2 Eagle 1.0 checkinId

# Submit the repository script to the remote server:
set script [::PackageUploader::createRepositoryScript "" $checkinId \
    "eagle" "1.0" "win64-x64" $files {}]
set cert [readFile [append $tmpScriptFileName ".asc"]]
::PackageUploader::submitPackageMetadata $apiKey MyPkg 1.2 Eagle $script $cert
```

## BUGS AND LIMITATIONS
1. **Fossil** only: Expects a Fossil SCM–based server. No Git, SVN, etc.
2. **Branch** merges: By default, this script sets/updates a special branch like `pkg_<name>_<version>`. Merging that branch back to trunk might need manual steps.
3. **Network** and **TLS** compile options in Eagle must be present.

## SEE ALSO
- **`pkgr.eagle`** — Repository client
- **`pkgd.eagle`** — Downloader
- **`fossil-scm.org`** — Fossil official site

## KEYWORDS
Uploader, Eagle, Tcl, Fossil, Packages, Signing, OpenPGP, Harpy

---

# 4. **common.tcl** — Common Tools for Native Tcl HTTP/HTTPS

## NAME
**common.tcl** — Shared HTTP and logging utilities for native Tcl

## SYNOPSIS
```tcl
source common.tcl
```
*(Often loaded automatically by `pkgd.eagle` or `pkgr.eagle` if running in **native Tcl** mode.)*

## DESCRIPTION
**common.tcl** is a script of **shared procedures** for **native Tcl** use. It does **not** load in Eagle. The code checks `[package present Eagle]` and errors if found. Its main tasks are:

1. **HTTP**: Safe HTTP GET, handling redirects, progress indicators, optional forced HTTPS.  
2. **TLS** fallback**: If `tls` is present, can forcibly upgrade `http://` to `https://`.  
3. **Logging**: Utilities like `pageLog`, `pageOut`, `pageProgress`.  
4. **Binary I/O**: e.g., `writeFile`, `makeBinaryChannel`.

## COMMANDS
Namespace: `::Eagle::Tools::Common` (though intended for pure Tcl usage).

1. **`getFileViaHttp uri redirectLimit channel quiet ?args?`**  
   - The main procedure for retrieving remote content with `[http::geturl]`.  
   - Honors variables like `forceSecureUri`, `mustHaveTls`, `allowInsecureRedirect`.
2. **`appendArgs args`**, **`writeFile fileName data`**  
   - Minor I/O helpers.
3. **`pageOut channel string`**, **`pageLog string`**  
   - Logging or progress output to the console/log system.
4. **`pageProgress channel type milliseconds`**  
   - Repeatedly writes a small progress character every `milliseconds`.
5. **`setupCommonVariables force`**  
   - Configures defaults (TLS usage, verbosity, timeouts).

## OPTIONS
- **`forceSecureUri`**: Boolean, forcibly convert `http://` → `https://` if `tls` is loaded.  
- **`mustHaveTls`**: Boolean, raise error if `tls` not available.  
- **`allowInsecureUri`**: Boolean, if true, can degrade to `http://`.  
- **`verboseGetUrl`**: Logs the `[http::geturl]` invocation details.

## EXAMPLES
```tcl
package require http
source common.tcl

# Download a file to memory:
set data [::Eagle::Tools::Common::getFileViaHttp \
    "https://example.com/file.txt" 5 stdout false -binary true]

# Write it out:
::Eagle::Tools::Common::writeFile "/tmp/file.txt" $data
```

## BUGS AND LIMITATIONS
1. **Only for Native Tcl**: Exits if `[package present Eagle]` succeeds.
2. **`tls`** versions differ across platforms: if `tls` is buggy or missing, set `allowInsecureUri`.
3. **No Automatic IPv6**: Forces IPv4 for Tcl 8.6 in certain conditions.

## SEE ALSO
- **`pkgd.eagle`**, **`pkgr.eagle`** which may load this in a native Tcl context.  
- **[TLS package](https://core.tcl-lang.org/tcltls/)**

## KEYWORDS
HTTP, HTTPS, Tcl, Download, Logging, TLS

---

# Final Note on Usage
Generally:
1. **`pkgr.eagle`** is the client that intercepts `[package unknown]` for auto-download.
2. **`pkgd.eagle`** is the lower-level downloader script (explicit).
3. **`pkgu.eagle`** is the uploader script for staging/committing with Fossil.
4. **`common.tcl`** provides shared HTTP/TLS progress logic (only for native Tcl).

These scripts are designed to **work together** to manage the complete lifecycle of **fetching**, **verifying**, and **publishing** packages for Eagle or Tcl. Each is documented with best-practice Tcl man-page style sections, as presented here.
Z 2b384a05b77c6abb2c719fe141770c32