CLI

Push a folder

One command loads a folder into a document base and sends only what changed. It is safe to interrupt, cheap to re-run, and it leaves no local state to go stale.

What push actually does

Six steps, in this order, and the order is the point:

1. Walk and hash. What is here, and what are its bytes. Local, no network.
2. Diff. Which of those the server does not already have. One small request.
3. Upload URLs. A write target for exactly that delta, never for unchanged files.
4. Upload. The bytes, in parallel, with retries, straight to storage.
5. Commit. Register them and open a sync run. This is the only step that changes anything.
6. Wait. Watch the run to a terminal state, with a progress bar.

Two properties fall out of that. It is resumable: nothing is registered until commit, so a run killed anywhere before it leaves the base exactly as it was, and re-running re-hashes, re-diffs and sends only what is still outstanding. And it is cheap on the tenth run: a folder of 50,000 documents where three changed is one diff request and three uploads.

engram push ./docs --corpus "Support KB"

Memory is bounded throughout. Hashing and uploads stream in chunks, so the peak is set by --concurrency, not by the size of your largest file.

What gets picked up

Push asks the platform what it can read rather than guessing, so a deployment that has learned a new file type picks it up without a CLI upgrade. The built-in fallback list, used when the server does not publish one, is .txt, .md, .pdf, .docx, .doc, .html, .htm, .xlsx, .xls, .csv and .tsv. When it falls back, it says so in the summary.

Files are skipped for exactly four reasons, and each is counted separately so the summary tells you which:

Globs match against the full relative path or the bare filename, whichever is unambiguous, so --exclude '*.tmp' means every .tmp anywhere and --include 'reports/**' means that folder. Both flags repeat.

engram push ./docs --corpus "Support KB" \
  --include '*.pdf' --include '*.docx' \
  --exclude 'drafts/**'

A document's identity is its path relative to the folder you pushed, with forward slashes, so the same folder pushed from Windows and from Linux produces identical keys and nothing reads as changed. Pushing a single file works too, and the document takes that file's name.

Skip files with .engramignore

Put a .engramignore file at the top of the folder to keep paths out of a document base for good, instead of spelling out --exclude on every command. It takes gitignore syntax:

# scratch space, never push this
drafts/
*.log
archive/2019/

Ignored files are never read, never hashed and never uploaded, and folders listed in it are skipped whole rather than walked first. It works alongside --include and --exclude, and hidden files and folders (anything starting with a dot) are skipped either way. The .engramignore file itself is never pushed.

The push summary counts what the file kept out, on a skipped (.engramignore) row, so a rule matching far more than you meant reads differently from a folder with nothing in it. A folder skipped whole counts as one rather than as the files inside it, because nothing ever looked inside — that is what skipping it is for. --verbose lists what matched, with a trailing slash on the folders, and --json carries the same list under ignored.

Google Docs, Sheets and Slides

Google files need one extra step before they can be pushed from a folder. If you sync Google Drive to your computer with Drive for Desktop, a Google Doc on disk is a shortcut, not a document: the file is a link a few hundred bytes long and holds none of the text. Pushing a folder of them would upload nothing.

engram push counts them and tells you. Add --verbose to list the files. Two ways to bring them in:

1. Export them to .docx or .pdf in Google Drive, then push again.
2. Or connect Google Drive in the app and pick the files there, which imports the live documents and keeps them up to date as they change. See Google Drive.

Look before you load

--dry-run does the walk, the hash and the diff, then stops and prints the plan. It changes nothing and needs no write permission on the base.

engram push ./docs --corpus "Support KB" --dry-run

DRY RUN: push /home/me/docs -> Support KB
files found                  412
skipped (file type)           37
skipped (empty)                2
in the manifest              373
new                           12
changed                        3
unchanged                    358
on the server but not here     5

new (12):
  policies/refunds-2026.md
  ...

It lists the first 20 paths in each of new, changed and missing, and says how many more there are. With --json, the full path lists ride along in the document, which is the answer to "what would this actually do".

Mirror mode

By default push never deletes: documents on the server that are no longer in your folder are reported as on the server but not here and left alone. That way a push from the wrong folder can never empty a document base.

--delete turns it into a mirror, removing what is gone:

engram push ./docs --corpus "Support KB" --delete --dry-run   # check first
engram push ./docs --corpus "Support KB" --delete

Combine it with --dry-run the first time and the summary says exactly how many documents --delete would remove, before it removes any.

Keep a folder in sync with --watch

Add --watch and the push stays open, sending each change as you make it. Good for a working folder you edit through the day, or a shared drive that a team keeps adding to.

engram push ./docs --corpus Sales --watch

It pushes once when it starts, then again every time the folder goes quiet after a change. Changes are grouped, so saving ten files at once is one push and not ten. Each push prints one line with what moved. Press Ctrl-C to stop.

Two seconds of quiet is the default. Raise it with --debounce 10 if you have something writing to the folder steadily and want fewer, larger pushes.

Watch mode is part of the CLI on every plan. It runs on your machine with your own key and does exactly what running engram push by hand does, so nothing is left behind if you close it. If your laptop drops off the network mid-push, the session prints the error and keeps watching, and the next change re-sends whatever did not land.

A note for OneDrive on Windows

If the folder is in OneDrive with Files On-Demand turned on, files that are not downloaded yet still report their full size, and anything that reads one pulls it down. Watching a large folder that has not been downloaded will therefore download it. Either point the push at a folder you have set to "Always keep on this device", or use --exclude to scope it to the part you actually work in.

Large loads and long waits

Four options shape a big push.

engram push ./archive --corpus "Support KB" --concurrency 24 --no-wait
engram status "Support KB"       # pick the story back up whenever

When the wait times out, the message says so plainly and points at engram status: the upload is committed and the server is still working. That is a different outcome from a failure, and the exit code and the message both reflect it.

Pushing from S3

An s3:// target reads your bucket with your own local AWS credentials, whatever boto3 already resolves, and streams the bytes up through your machine:

pipx install "engram-dynamics[s3]"
engram push s3://acme-docs/reports --corpus "Support KB" --region us-east-1

Objects in Glacier and Deep Archive storage classes are skipped rather than listed and then failed one by one. Console-created folder markers are ignored.

Reading the summary, and failures

Every push ends with one table. Numbers live in that table only, so nothing in the output can disagree with itself:

push /home/me/docs -> Support KB
files found                  412
in the manifest              373
new                           12
changed                        3
unchanged                    358
uploaded                      15
uploaded bytes            8.4 MB
committed                    373

sync run  r-77c1e0
state     succeeded

If some files fail to upload, nothing is committed. Committing a partial upload would register documents whose bytes are not all there, so instead the command prints what it did, names the failures, and tells you to re-run the same command: the files that did upload will not be sent again.

Push retries an upload on the statuses that mean "later, not never" (408, 425, 429, 500, 502, 503 and 504), three attempts with a backoff. A 403 on an expired signature or a 400 on a bad request fails immediately, because retrying it only wastes your time before telling you the truth.

Next