D 2025-01-10T06:05:44.753
L Documentation
N text/x-markdown
P 688196daf451f1ea42b1a03254a5a74b9959ee84
U mistachkin
W 16217
Preliminary documentation for <b>v1</b> of the Package Client is available <b>[/doc/trunk/doc/v1.html|here]</b>.
---
Below is a set of **four man-page–style documents** covering each core script in the Eagle “package client toolset.” They follow a **Tcl SDK–style** documentation format, with sections like **Name**, **Synopsis**, **Description**, **Commands**, **Options**, **Examples**, **Bugs/Limitations**, **See Also**, and **Keywords**. Each script—**`pkgr.eagle`**, **`pkgd.eagle`**, **`pkgu.eagle`**, and **`common.tcl`**—is documented separately.
---
# 1. **pkgr.eagle** — Package Repository Client
## NAME
**pkgr.eagle** — Eagle Package Repository Client
## SYNOPSIS
```tcl
source pkgr.eagle
```
*(Assumes you have placed `pkgr.eagle` in your script library or added its directory to `::auto_path`. This script can be loaded automatically by `package require Eagle.Package.Repository`.)*
## DESCRIPTION
The **pkgr.eagle** script provides a client interface to the remote Eagle **package repository**. It can:
1. **Hook** into Tcl’s or Eagle’s `[package unknown]` mechanism, so that missing packages are fetched automatically from a remote server.
2. **Download** package metadata, scripts, and signatures using either `[uri download]` (Eagle) or `[::http::geturl]` (Tcl).
3. **Verify** package scripts and certificates (OpenPGP or Harpy).
4. **Offer** a fallback for anonymous vs. API-key–based access.
In short, it helps your application load packages on demand from a central repository, with security checks and minimal local overhead.
### Primary Features
- **Inline** hooking of `[package unknown]`: If a package is not found locally, `pkgr.eagle` queries the remote server, downloads the script, verifies signatures, and registers it in `[package ifneeded ...]`.
- **OpenPGP verification**: Ensures scripts are signed with GPG/gnupg (for example, `gpg2`).
- **Harpy certificates**: Supports advanced script signing with Harpy (for `.eagle` scripts).
- **Namespace**: All functionality is in `::PackageRepository`.
## COMMANDS
Below is a summary of the primary commands provided (or influenced) by **pkgr.eagle**. Most live under the `::PackageRepository` namespace.
1. **`setupRepositoryServerVars force`**
Initializes or resets server URN variables for package lookups (e.g. `lookupUrn`, `submitUrn`, etc.).
2. **`setupRepositoryUriVars force`**
Initializes or resets URI variables (e.g. `baseUri`, `lookupUri`, etc.).
3. **`setupPackageUnknownVars force`**
Sets up defaults (API keys, hooking preferences, verbosity) for the `[package unknown]` mechanism.
4. **`setupPackageUnknownHandler`**
Optionally installs the custom `[package unknown]` handler that automatically fetches missing packages from the repository.
5. **`packageUnknownHandler package ?version?`**
The actual `[package unknown]` hook. Called by Tcl/Eagle if a requested package is not found locally. In turn, it attempts `getPackageFromRepository`.
6. **`getPackageFromRepository package version caller`**
The central routine that requests metadata from the remote server, verifies the script’s signature, and registers the package.
7. **`processLookupMetadata metadataVarName`**
Given extracted metadata (script, certificate), verifies and loads them, respecting either “Tcl” or “Eagle” language.
8. **`extractAndVerifyLookupMetadata result varName caller`**
Parses the raw repository response, checks name/patchLevel/language/script/certificate, and stores them into `varName`.
9. **`probeForOpenPgpInstallation`, `openPgpMustBeInstalled`**
Check for GPG/gnupg presence and appropriate version.
10. **`verifyOpenPgpSignature fileName`**
Returns a boolean indicating if the specified OpenPGP signature file is valid.
... and many more utility procedures like `isEagle`, `eagleMustBeReady`, `makeEagleReady`, `stringIsList`, `formatResult`, etc.
## OPTIONS
- **`autoHook`**: If true, automatically hooks `[package unknown]`.
- **`verboseUnknownResult`**: If true, logs debug info each time `[package unknown]` is invoked.
- **`forceSecureUri`**: If true, upgrades `http://` to `https://` when TLS is available.
- **`mustHaveTls`**: If true, raises errors if `tls` cannot be loaded.
- **`strictUnknownLanguage`**: If true, disallows loading a package if its declared language doesn’t match the current interpreter.
## EXAMPLES
```tcl
# Basic usage in Eagle:
package require Eagle.Package.Repository
# Automatic hooking:
::PackageRepository::setupPackageUnknownVars false
::PackageRepository::setupPackageUnknownHandler
# Now [package require SomeRemotePackage] will fetch from the remote repo.
```
## BUGS AND LIMITATIONS
1. **Strict security**: By default, unknown scripts must have valid signatures (OpenPGP or Harpy). This can fail if `gpg` is not installed or if the signature is missing/invalid.
2. **Non-automatic**: While it can be automatic, you must ensure your environment is configured properly (`auto_path`, hooking, etc.).
3. **Limited OS checks**: Some logic (like IPv6 vs. IPv4 forcing) is partial and may not handle all edge cases.
## SEE ALSO
- **`pkgd.eagle`** — Package Downloader
- **`pkgu.eagle`** — Package Uploader
- **`common.tcl`** — Shared HTTP and file utilities
- **[Fossil SCM](https://fossil-scm.org/)** if using the “file repository” approach
## KEYWORDS
Packages, Repository, Eagle, Tcl, Download, `[package unknown]`, OpenPGP, Harpy, Signing
---
# 2. **pkgd.eagle** — Package Downloader Client
## NAME
**pkgd.eagle** — Eagle Package Downloader Client
## SYNOPSIS
```tcl
source pkgd.eagle
```
*(Often invoked via `package require Eagle.Package.Downloader`.)*
## DESCRIPTION
**pkgd.eagle** focuses on **downloading** package files from a remote server, optionally persisting them, verifying signatures, and bridging them into **Tcl** or **Eagle** interpreters. It also handles some multi-language awareness: e.g., an Eagle script can fetch `.tcl` packages and vice versa (with the help of *Garuda* on Tcl or *tclMustBeReady* in Eagle).
### Key Capabilities
1. **Download**: Using `getPackageFile` or `downloadFiles` with either standard `[http]` or `[uri download]`.
2. **Signature checks**: OpenPGP or Harpy.
3. **Persistent vs. Temporary**: Files can be placed in a permanent directory or a temporary one.
4. **Auto-path** modifications**: If `-useAutoPath` is true, newly downloaded directories get appended to the auto-path.
## COMMANDS
All commands reside in `::PackageDownloader`:
1. **`downloadFiles language version platform fileNames options`**
Downloads a list of files from the package server.
- `-persistent`: store them permanently.
- `-usePgp`: enable signature verification.
- `-useAutoPath`: add directories containing the downloaded files to `::auto_path`.
2. **`downloadOneFile language version platform fileName localFileName usePgp`**
A low-level routine for fetching a single file and verifying if `usePgp` is set.
3. **`resetCookieAndLogin userName password`** / **`logoutAndResetCookie`**
Handles session cookies for private or public logins to the remote file server.
4. **`probeForOpenPgpInstallation`**, **`openPgpMustBeInstalled`**
Similar to pkgr.eagle, ensures GPG is present.
5. **`maybeAddToAutoPath language directory`**
Appends a directory to the auto-path for the specified language (Tcl or Eagle) if it’s not already present.
6. **`getLookupData`, `extractAndVerifyLookupMetadata`,** etc.
Utility procedures for internal or advanced usage.
## OPTIONS
- **`-persistent`**: Boolean. If true, downloaded files go into the “persistentRootDirectory.”
- **`-useAutoPath`**: Boolean. If true, automatically add the downloaded directory to `[auto_path]`.
- **`-usePgp`**: Boolean. If true, require an OpenPGP signature for each file.
- **`-allowUpdate`**: Boolean. If true, overwrites existing package files.
## EXAMPLES
```tcl
package require Eagle.Package.Downloader
# Download a single file in Eagle:
::PackageDownloader::downloadOneFile eagle 1.0 neutral \
mypkg.eagle /tmp/mypkg.eagle true
# Download multiple, persist, and auto-path them:
::PackageDownloader::downloadFiles tcl 8.6 "" {pkgIndex.tcl README.txt} {
-persistent true
-useAutoPath true
-usePgp true
}
```
## BUGS AND LIMITATIONS
1. **Cross-language bridging**: Relies on Eagle’s `[tclMustBeReady]` or Tcl’s `[eagleMustBeReady]`.
2. **File collisions**: If `-allowUpdate` is `false`, existing files cause errors.
3. **Security**: Temp directories must be local and not shared with untrusted processes.
## SEE ALSO
- **`pkgr.eagle`** — Repository Client (handles `[package unknown]`)
- **`pkgu.eagle`** — Uploader
- **`common.tcl`** — Shared utility code
## KEYWORDS
Downloader, Eagle, Tcl, Packages, HTTP, GPG, auto-path
---
# 3. **pkgu.eagle** — Package Uploader Client
## NAME
**pkgu.eagle** — Eagle Package Uploader Client
## SYNOPSIS
```tcl
source pkgu.eagle
```
*(Often invoked via `package require Eagle.Package.Uploader`.)*
## DESCRIPTION
**pkgu.eagle** manages **uploading** (staging and committing) new packages into a Fossil-based “package file server” and then **submitting** metadata to the remote repository. It expects:
- **Local** Fossil checkout of the “package file server” repository.
- **Proper** signings (OpenPGP and optionally Harpy for `.eagle` scripts).
- **Connectivity** to the same server used by `pkgr.eagle` and `pkgd.eagle`.
### Key Capabilities
1. **Stage new package files** in the local Fossil checkout (`stagePackageFiles`).
2. **Commit** them with a custom branch name (`commitPackageFiles`).
3. **Generate** a repository script that references those newly committed files.
4. **Submit** the script and signature to the remote server (`submitPackageMetadata`).
## COMMANDS
Namespace: `::PackageUploader`
1. **`stagePackageFiles language version platform fileNames`**
- Verifies no local pending changes, correct project/branch.
- Copies the given files into `packages/<lang>/<version>/<platform>` in the Fossil checkout.
- Runs `fossil add`, signs them with OpenPGP/Harpy.
2. **`commitPackageFiles package patchLevel language version varName`**
- Fossil commit with a message referencing the package name, patch level, language, and local checkout ID.
- If success, stores the new checkin ID in `varName`.
3. **`createRepositoryScript serverId versionId language version platform fileNames options`**
- Produces an inline Eagle script that can be used by the repository client to download these newly staged files.
4. **`submitPackageMetadata apiKey package patchLevel language script certificate`**
- POSTs a multipart form to the repository’s “submit” endpoint, telling it about the newly available package version.
5. **`getCheckoutDirectory`, `verifyCheckoutDirectory`**
- Ensures the local Fossil repo is present and valid.
6. **`fossilMustBeInstalled`, `verifyThereAreNoChanges`, `verifyThisIsTheCorrectProject`, `verifyThisIsTheCorrectBranch`**
- Housekeeping checks for Fossil usage.
## OPTIONS
- **`projectCode`**: The known project code for the Fossil repo. If mismatched, `verifyThisIsTheCorrectProject` fails.
- **`fossilFileNameOnly`**: Name of the Fossil CLI executable, default `fossil.exe` or `fossil`.
- **`scriptDirectory`**: Directory containing the package client tools; used to detect the local checkout.
## EXAMPLES
```tcl
package require Eagle.Package.Uploader
# Stage & commit a new package version:
set files [list /path/to/MyPkg.eagle /path/to/pkgIndex.eagle]
::PackageUploader::stagePackageFiles eagle 1.0 win64-x64 $files
::PackageUploader::commitPackageFiles MyPkg 1.2 Eagle 1.0 checkinId
# Submit the repository script to the remote server:
set script [::PackageUploader::createRepositoryScript "" $checkinId \
"eagle" "1.0" "win64-x64" $files {}]
set cert [readFile [append $tmpScriptFileName ".asc"]]
::PackageUploader::submitPackageMetadata $apiKey MyPkg 1.2 Eagle $script $cert
```
## BUGS AND LIMITATIONS
1. **Fossil** only: Expects a Fossil SCM–based server. No Git, SVN, etc.
2. **Branch** merges: By default, this script sets/updates a special branch like `pkg_<name>_<version>`. Merging that branch back to trunk might need manual steps.
3. **Network** and **TLS** compile options in Eagle must be present.
## SEE ALSO
- **`pkgr.eagle`** — Repository client
- **`pkgd.eagle`** — Downloader
- **`fossil-scm.org`** — Fossil official site
## KEYWORDS
Uploader, Eagle, Tcl, Fossil, Packages, Signing, OpenPGP, Harpy
---
# 4. **common.tcl** — Common Tools for Native Tcl HTTP/HTTPS
## NAME
**common.tcl** — Shared HTTP and logging utilities for native Tcl
## SYNOPSIS
```tcl
source common.tcl
```
*(Often loaded automatically by `pkgd.eagle` or `pkgr.eagle` if running in **native Tcl** mode.)*
## DESCRIPTION
**common.tcl** is a script of **shared procedures** for **native Tcl** use. It does **not** load in Eagle. The code checks `[package present Eagle]` and errors if found. Its main tasks are:
1. **HTTP**: Safe HTTP GET, handling redirects, progress indicators, optional forced HTTPS.
2. **TLS** fallback**: If `tls` is present, can forcibly upgrade `http://` to `https://`.
3. **Logging**: Utilities like `pageLog`, `pageOut`, `pageProgress`.
4. **Binary I/O**: e.g., `writeFile`, `makeBinaryChannel`.
## COMMANDS
Namespace: `::Eagle::Tools::Common` (though intended for pure Tcl usage).
1. **`getFileViaHttp uri redirectLimit channel quiet ?args?`**
- The main procedure for retrieving remote content with `[http::geturl]`.
- Honors variables like `forceSecureUri`, `mustHaveTls`, `allowInsecureRedirect`.
2. **`appendArgs args`**, **`writeFile fileName data`**
- Minor I/O helpers.
3. **`pageOut channel string`**, **`pageLog string`**
- Logging or progress output to the console/log system.
4. **`pageProgress channel type milliseconds`**
- Repeatedly writes a small progress character every `milliseconds`.
5. **`setupCommonVariables force`**
- Configures defaults (TLS usage, verbosity, timeouts).
## OPTIONS
- **`forceSecureUri`**: Boolean, forcibly convert `http://` → `https://` if `tls` is loaded.
- **`mustHaveTls`**: Boolean, raise error if `tls` not available.
- **`allowInsecureUri`**: Boolean, if true, can degrade to `http://`.
- **`verboseGetUrl`**: Logs the `[http::geturl]` invocation details.
## EXAMPLES
```tcl
package require http
source common.tcl
# Download a file to memory:
set data [::Eagle::Tools::Common::getFileViaHttp \
"https://example.com/file.txt" 5 stdout false -binary true]
# Write it out:
::Eagle::Tools::Common::writeFile "/tmp/file.txt" $data
```
## BUGS AND LIMITATIONS
1. **Only for Native Tcl**: Exits if `[package present Eagle]` succeeds.
2. **`tls`** versions differ across platforms: if `tls` is buggy or missing, set `allowInsecureUri`.
3. **No Automatic IPv6**: Forces IPv4 for Tcl 8.6 in certain conditions.
## SEE ALSO
- **`pkgd.eagle`**, **`pkgr.eagle`** which may load this in a native Tcl context.
- **[TLS package](https://core.tcl-lang.org/tcltls/)**
## KEYWORDS
HTTP, HTTPS, Tcl, Download, Logging, TLS
---
# Final Note on Usage
Generally:
1. **`pkgr.eagle`** is the client that intercepts `[package unknown]` for auto-download.
2. **`pkgd.eagle`** is the lower-level downloader script (explicit).
3. **`pkgu.eagle`** is the uploader script for staging/committing with Fossil.
4. **`common.tcl`** provides shared HTTP/TLS progress logic (only for native Tcl).
These scripts are designed to **work together** to manage the complete lifecycle of **fetching**, **verifying**, and **publishing** packages for Eagle or Tcl. Each is documented with best-practice Tcl man-page style sections, as presented here.
Z 2b384a05b77c6abb2c719fe141770c32