1
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
|
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
|
Preliminary documentation for <b>v1</b> of the Package Client is available <b>[/doc/trunk/doc/v1.html|here]</b>.
---
Below is a set of **four man-page–style documents** covering each core script in the Eagle “package client toolset.” They follow a **Tcl SDK–style** documentation format, with sections like **Name**, **Synopsis**, **Description**, **Commands**, **Options**, **Examples**, **Bugs/Limitations**, **See Also**, and **Keywords**. Each script—**`pkgr.eagle`**, **`pkgd.eagle`**, **`pkgu.eagle`**, and **`common.tcl`**—is documented separately.
---
# 1. **pkgr.eagle** — Package Repository Client
## NAME
**pkgr.eagle** — Eagle Package Repository Client
## SYNOPSIS
```tcl
source pkgr.eagle
```
*(Assumes you have placed `pkgr.eagle` in your script library or added its directory to `::auto_path`. This script can be loaded automatically by `package require Eagle.Package.Repository`.)*
## DESCRIPTION
The **pkgr.eagle** script provides a client interface to the remote Eagle **package repository**. It can:
1. **Hook** into Tcl’s or Eagle’s `[package unknown]` mechanism, so that missing packages are fetched automatically from a remote server.
2. **Download** package metadata, scripts, and signatures using either `[uri download]` (Eagle) or `[::http::geturl]` (Tcl).
3. **Verify** package scripts and certificates (OpenPGP or Harpy).
4. **Offer** a fallback for anonymous vs. API-key–based access.
In short, it helps your application load packages on demand from a central repository, with security checks and minimal local overhead.
### Primary Features
- **Inline** hooking of `[package unknown]`: If a package is not found locally, `pkgr.eagle` queries the remote server, downloads the script, verifies signatures, and registers it in `[package ifneeded ...]`.
- **OpenPGP verification**: Ensures scripts are signed with GPG/gnupg (for example, `gpg2`).
- **Harpy certificates**: Supports advanced script signing with Harpy (for `.eagle` scripts).
- **Namespace**: All functionality is in `::PackageRepository`.
## COMMANDS
Below is a summary of the primary commands provided (or influenced) by **pkgr.eagle**. Most live under the `::PackageRepository` namespace.
1. **`setupRepositoryServerVars force`**
Initializes or resets server URN variables for package lookups (e.g. `lookupUrn`, `submitUrn`, etc.).
2. **`setupRepositoryUriVars force`**
Initializes or resets URI variables (e.g. `baseUri`, `lookupUri`, etc.).
3. **`setupPackageUnknownVars force`**
Sets up defaults (API keys, hooking preferences, verbosity) for the `[package unknown]` mechanism.
4. **`setupPackageUnknownHandler`**
Optionally installs the custom `[package unknown]` handler that automatically fetches missing packages from the repository.
5. **`packageUnknownHandler package ?version?`**
The actual `[package unknown]` hook. Called by Tcl/Eagle if a requested package is not found locally. In turn, it attempts `getPackageFromRepository`.
6. **`getPackageFromRepository package version caller`**
The central routine that requests metadata from the remote server, verifies the script’s signature, and registers the package.
7. **`processLookupMetadata metadataVarName`**
Given extracted metadata (script, certificate), verifies and loads them, respecting either “Tcl” or “Eagle” language.
8. **`extractAndVerifyLookupMetadata result varName caller`**
Parses the raw repository response, checks name/patchLevel/language/script/certificate, and stores them into `varName`.
9. **`probeForOpenPgpInstallation`, `openPgpMustBeInstalled`**
Check for GPG/gnupg presence and appropriate version.
10. **`verifyOpenPgpSignature fileName`**
Returns a boolean indicating if the specified OpenPGP signature file is valid.
... and many more utility procedures like `isEagle`, `eagleMustBeReady`, `makeEagleReady`, `stringIsList`, `formatResult`, etc.
## OPTIONS
- **`autoHook`**: If true, automatically hooks `[package unknown]`.
- **`verboseUnknownResult`**: If true, logs debug info each time `[package unknown]` is invoked.
- **`forceSecureUri`**: If true, upgrades `http://` to `https://` when TLS is available.
- **`mustHaveTls`**: If true, raises errors if `tls` cannot be loaded.
- **`strictUnknownLanguage`**: If true, disallows loading a package if its declared language doesn’t match the current interpreter.
## EXAMPLES
```tcl
# Basic usage in Eagle:
package require Eagle.Package.Repository
# Automatic hooking:
::PackageRepository::setupPackageUnknownVars false
::PackageRepository::setupPackageUnknownHandler
# Now [package require SomeRemotePackage] will fetch from the remote repo.
```
## BUGS AND LIMITATIONS
1. **Strict security**: By default, unknown scripts must have valid signatures (OpenPGP or Harpy). This can fail if `gpg` is not installed or if the signature is missing/invalid.
2. **Non-automatic**: While it can be automatic, you must ensure your environment is configured properly (`auto_path`, hooking, etc.).
3. **Limited OS checks**: Some logic (like IPv6 vs. IPv4 forcing) is partial and may not handle all edge cases.
## SEE ALSO
- **`pkgd.eagle`** — Package Downloader
- **`pkgu.eagle`** — Package Uploader
- **`common.tcl`** — Shared HTTP and file utilities
- **[Fossil SCM](https://fossil-scm.org/)** if using the “file repository” approach
## KEYWORDS
Packages, Repository, Eagle, Tcl, Download, `[package unknown]`, OpenPGP, Harpy, Signing
---
# 2. **pkgd.eagle** — Package Downloader Client
## NAME
**pkgd.eagle** — Eagle Package Downloader Client
## SYNOPSIS
```tcl
source pkgd.eagle
```
*(Often invoked via `package require Eagle.Package.Downloader`.)*
## DESCRIPTION
**pkgd.eagle** focuses on **downloading** package files from a remote server, optionally persisting them, verifying signatures, and bridging them into **Tcl** or **Eagle** interpreters. It also handles some multi-language awareness: e.g., an Eagle script can fetch `.tcl` packages and vice versa (with the help of *Garuda* on Tcl or *tclMustBeReady* in Eagle).
### Key Capabilities
1. **Download**: Using `getPackageFile` or `downloadFiles` with either standard `[http]` or `[uri download]`.
2. **Signature checks**: OpenPGP or Harpy.
3. **Persistent vs. Temporary**: Files can be placed in a permanent directory or a temporary one.
4. **Auto-path** modifications**: If `-useAutoPath` is true, newly downloaded directories get appended to the auto-path.
## COMMANDS
All commands reside in `::PackageDownloader`:
1. **`downloadFiles language version platform fileNames options`**
Downloads a list of files from the package server.
- `-persistent`: store them permanently.
- `-usePgp`: enable signature verification.
- `-useAutoPath`: add directories containing the downloaded files to `::auto_path`.
2. **`downloadOneFile language version platform fileName localFileName usePgp`**
A low-level routine for fetching a single file and verifying if `usePgp` is set.
3. **`resetCookieAndLogin userName password`** / **`logoutAndResetCookie`**
Handles session cookies for private or public logins to the remote file server.
4. **`probeForOpenPgpInstallation`**, **`openPgpMustBeInstalled`**
Similar to pkgr.eagle, ensures GPG is present.
5. **`maybeAddToAutoPath language directory`**
Appends a directory to the auto-path for the specified language (Tcl or Eagle) if it’s not already present.
6. **`getLookupData`, `extractAndVerifyLookupMetadata`,** etc.
Utility procedures for internal or advanced usage.
## OPTIONS
- **`-persistent`**: Boolean. If true, downloaded files go into the “persistentRootDirectory.”
- **`-useAutoPath`**: Boolean. If true, automatically add the downloaded directory to `[auto_path]`.
- **`-usePgp`**: Boolean. If true, require an OpenPGP signature for each file.
- **`-allowUpdate`**: Boolean. If true, overwrites existing package files.
## EXAMPLES
```tcl
package require Eagle.Package.Downloader
# Download a single file in Eagle:
::PackageDownloader::downloadOneFile eagle 1.0 neutral \
mypkg.eagle /tmp/mypkg.eagle true
# Download multiple, persist, and auto-path them:
::PackageDownloader::downloadFiles tcl 8.6 "" {pkgIndex.tcl README.txt} {
-persistent true
-useAutoPath true
-usePgp true
}
```
## BUGS AND LIMITATIONS
1. **Cross-language bridging**: Relies on Eagle’s `[tclMustBeReady]` or Tcl’s `[eagleMustBeReady]`.
2. **File collisions**: If `-allowUpdate` is `false`, existing files cause errors.
3. **Security**: Temp directories must be local and not shared with untrusted processes.
## SEE ALSO
- **`pkgr.eagle`** — Repository Client (handles `[package unknown]`)
- **`pkgu.eagle`** — Uploader
- **`common.tcl`** — Shared utility code
## KEYWORDS
Downloader, Eagle, Tcl, Packages, HTTP, GPG, auto-path
---
# 3. **pkgu.eagle** — Package Uploader Client
## NAME
**pkgu.eagle** — Eagle Package Uploader Client
## SYNOPSIS
```tcl
source pkgu.eagle
```
*(Often invoked via `package require Eagle.Package.Uploader`.)*
## DESCRIPTION
**pkgu.eagle** manages **uploading** (staging and committing) new packages into a Fossil-based “package file server” and then **submitting** metadata to the remote repository. It expects:
- **Local** Fossil checkout of the “package file server” repository.
- **Proper** signings (OpenPGP and optionally Harpy for `.eagle` scripts).
- **Connectivity** to the same server used by `pkgr.eagle` and `pkgd.eagle`.
### Key Capabilities
1. **Stage new package files** in the local Fossil checkout (`stagePackageFiles`).
2. **Commit** them with a custom branch name (`commitPackageFiles`).
3. **Generate** a repository script that references those newly committed files.
4. **Submit** the script and signature to the remote server (`submitPackageMetadata`).
## COMMANDS
Namespace: `::PackageUploader`
1. **`stagePackageFiles language version platform fileNames`**
- Verifies no local pending changes, correct project/branch.
- Copies the given files into `packages/<lang>/<version>/<platform>` in the Fossil checkout.
- Runs `fossil add`, signs them with OpenPGP/Harpy.
2. **`commitPackageFiles package patchLevel language version varName`**
- Fossil commit with a message referencing the package name, patch level, language, and local checkout ID.
- If success, stores the new checkin ID in `varName`.
3. **`createRepositoryScript serverId versionId language version platform fileNames options`**
- Produces an inline Eagle script that can be used by the repository client to download these newly staged files.
4. **`submitPackageMetadata apiKey package patchLevel language script certificate`**
- POSTs a multipart form to the repository’s “submit” endpoint, telling it about the newly available package version.
5. **`getCheckoutDirectory`, `verifyCheckoutDirectory`**
- Ensures the local Fossil repo is present and valid.
6. **`fossilMustBeInstalled`, `verifyThereAreNoChanges`, `verifyThisIsTheCorrectProject`, `verifyThisIsTheCorrectBranch`**
- Housekeeping checks for Fossil usage.
## OPTIONS
- **`projectCode`**: The known project code for the Fossil repo. If mismatched, `verifyThisIsTheCorrectProject` fails.
- **`fossilFileNameOnly`**: Name of the Fossil CLI executable, default `fossil.exe` or `fossil`.
- **`scriptDirectory`**: Directory containing the package client tools; used to detect the local checkout.
## EXAMPLES
```tcl
package require Eagle.Package.Uploader
# Stage & commit a new package version:
set files [list /path/to/MyPkg.eagle /path/to/pkgIndex.eagle]
::PackageUploader::stagePackageFiles eagle 1.0 win64-x64 $files
::PackageUploader::commitPackageFiles MyPkg 1.2 Eagle 1.0 checkinId
# Submit the repository script to the remote server:
set script [::PackageUploader::createRepositoryScript "" $checkinId \
"eagle" "1.0" "win64-x64" $files {}]
set cert [readFile [append $tmpScriptFileName ".asc"]]
::PackageUploader::submitPackageMetadata $apiKey MyPkg 1.2 Eagle $script $cert
```
## BUGS AND LIMITATIONS
1. **Fossil** only: Expects a Fossil SCM–based server. No Git, SVN, etc.
2. **Branch** merges: By default, this script sets/updates a special branch like `pkg_<name>_<version>`. Merging that branch back to trunk might need manual steps.
3. **Network** and **TLS** compile options in Eagle must be present.
## SEE ALSO
- **`pkgr.eagle`** — Repository client
- **`pkgd.eagle`** — Downloader
- **`fossil-scm.org`** — Fossil official site
## KEYWORDS
Uploader, Eagle, Tcl, Fossil, Packages, Signing, OpenPGP, Harpy
---
# 4. **common.tcl** — Common Tools for Native Tcl HTTP/HTTPS
## NAME
**common.tcl** — Shared HTTP and logging utilities for native Tcl
## SYNOPSIS
```tcl
source common.tcl
```
*(Often loaded automatically by `pkgd.eagle` or `pkgr.eagle` if running in **native Tcl** mode.)*
## DESCRIPTION
**common.tcl** is a script of **shared procedures** for **native Tcl** use. It does **not** load in Eagle. The code checks `[package present Eagle]` and errors if found. Its main tasks are:
1. **HTTP**: Safe HTTP GET, handling redirects, progress indicators, optional forced HTTPS.
2. **TLS** fallback**: If `tls` is present, can forcibly upgrade `http://` to `https://`.
3. **Logging**: Utilities like `pageLog`, `pageOut`, `pageProgress`.
4. **Binary I/O**: e.g., `writeFile`, `makeBinaryChannel`.
## COMMANDS
Namespace: `::Eagle::Tools::Common` (though intended for pure Tcl usage).
1. **`getFileViaHttp uri redirectLimit channel quiet ?args?`**
- The main procedure for retrieving remote content with `[http::geturl]`.
- Honors variables like `forceSecureUri`, `mustHaveTls`, `allowInsecureRedirect`.
2. **`appendArgs args`**, **`writeFile fileName data`**
- Minor I/O helpers.
3. **`pageOut channel string`**, **`pageLog string`**
- Logging or progress output to the console/log system.
4. **`pageProgress channel type milliseconds`**
- Repeatedly writes a small progress character every `milliseconds`.
5. **`setupCommonVariables force`**
- Configures defaults (TLS usage, verbosity, timeouts).
## OPTIONS
- **`forceSecureUri`**: Boolean, forcibly convert `http://` → `https://` if `tls` is loaded.
- **`mustHaveTls`**: Boolean, raise error if `tls` not available.
- **`allowInsecureUri`**: Boolean, if true, can degrade to `http://`.
- **`verboseGetUrl`**: Logs the `[http::geturl]` invocation details.
## EXAMPLES
```tcl
package require http
source common.tcl
# Download a file to memory:
set data [::Eagle::Tools::Common::getFileViaHttp \
"https://example.com/file.txt" 5 stdout false -binary true]
# Write it out:
::Eagle::Tools::Common::writeFile "/tmp/file.txt" $data
```
## BUGS AND LIMITATIONS
1. **Only for Native Tcl**: Exits if `[package present Eagle]` succeeds.
2. **`tls`** versions differ across platforms: if `tls` is buggy or missing, set `allowInsecureUri`.
3. **No Automatic IPv6**: Forces IPv4 for Tcl 8.6 in certain conditions.
## SEE ALSO
- **`pkgd.eagle`**, **`pkgr.eagle`** which may load this in a native Tcl context.
- **[TLS package](https://core.tcl-lang.org/tcltls/)**
## KEYWORDS
HTTP, HTTPS, Tcl, Download, Logging, TLS
---
# Final Note on Usage
Generally:
1. **`pkgr.eagle`** is the client that intercepts `[package unknown]` for auto-download.
2. **`pkgd.eagle`** is the lower-level downloader script (explicit).
3. **`pkgu.eagle`** is the uploader script for staging/committing with Fossil.
4. **`common.tcl`** provides shared HTTP/TLS progress logic (only for native Tcl).
These scripts are designed to **work together** to manage the complete lifecycle of **fetching**, **verifying**, and **publishing** packages for Eagle or Tcl. Each is documented with best-practice Tcl man-page style sections, as presented here.
|