Nomad
Resumable SharePoint Online migration that never touches the sync client.
- Python
- Microsoft 365
Problem
A technician remoted into a client machine needs to move a large data set into SharePoint Online. Browser upload stalls, the OneDrive sync client gets stuck with opaque errors, and File Explorer over WebDAV gives up past a few gigabytes. The usual workaround, splitting the data by hand, still fails and explains nothing.
Approach
A command-line tool that writes server-side through the Microsoft Graph API using app-only authentication. Uploads are chunked and retried per-chunk, job state lives in SQLite so an interrupted run resumes, and a pre-flight scan catches the names SharePoint will reject before the transfer starts.
Outcome
v0.1.0 alpha, tagged and changelogged. Parallel uploads, resumable jobs, a per-job event log and summary report, an optional desktop console, and a Sites.Selected least-privilege path.
Nomad moves local file data into SharePoint Online and tells you the truth about what happened. When a transfer fails, you can see which file failed and why.
The sync client is the problem
The common way to move data into SharePoint is to drop it in a synced folder and let the OneDrive client push it up. That turns a migration into a sync operation and inherits every problem sync has, and those problems get worse in a tenant where many users are already syncing.
- Conflict file churn. A synced folder is constantly reconciled against the server copy, so any race produces conflict copies that someone then has to clean up by hand.
- Sync engine stalls. The "processing changes" state is a well-known dead end, and it offers no useful insight into what is stuck.
- Machine and identity coupling. The upload is chained to one user's machine with their OneDrive signed in and healthy. It cannot run unattended from anywhere else.
- Local staging cost. Pouring gigabytes into a synced location to stage a migration starts its own sync storm before anything reaches the destination.
Nomad does not use the sync client at all. It writes server-side through the Graph API, straight into the destination library, with app-only authentication. There is no synced folder, no placeholder file, and no OneDrive client anywhere in the transfer path, so that entire class of problem is gone by construction rather than managed around.
What this does not change is tenant throttling. Graph and SharePoint apply
limits per tenant, and a migration draws on the same budget as every active sync
client, so a large job still belongs outside business hours. That is where
resumable state earns its keep: an unattended overnight run picks up where it
left off if it is throttled, interrupted, or paused. Nomad honors Retry-After
on a 429, so it slows down under load instead of adding to the problem.
Find the failures before the transfer
nomad scan walks the source first and reports every problem with a stable code
and a severity. Errors block the migration unless you explicitly skip them;
warnings are informational.
It catches the things that reliably break a migration three hours in: characters SharePoint rejects, leading or trailing spaces, trailing periods, reserved device names, Office lock files, names over 256 characters, relative paths over 400, files past the per file size limit, and directories it could not read. Empty files and empty directories are flagged as warnings, since SharePoint does not store empty folders.
Resumable by design
Job state lives in SQLite, not in the process. A job that is interrupted is marked interrupted and resumes later by label, from either the command line or the desktop console. Uploads run in parallel across files while each file uploads its chunks serially, and the concurrency level is persisted on the job so a resume continues at the same setting.
Every job writes an event log and a final summary a technician can read.
Least privilege
App-only authentication with a certificate. For sensitive tenants, Nomad
supports Sites.Selected, which grants access to the specific sites you name
rather than every site in the tenant.
Requirements
Windows 11 or Windows Server 2019 or 2022, Python 3.11 or newer, and PowerShell
7.x as the recommended terminal. Nomad installs and its tests run on other
platforms, but the encrypted settings store relies on Windows DPAPI. An optional
desktop console installs with the gui extra and wraps the operating system's
built-in web view, so there is no Node, Electron, or separate browser to
install.
Roadmap
Tagged 0.1.0 with parallel resumable uploads and a pre-flight scan; validation at real scale is the gate to 1.0.
Full roadmap →