megfile

Provide unified Python file access across local, S3, SFTP, WebDAV, HTTP, and HDFS backends.

176|20|Updated Aug 23, 2021
One-click install
npx skills add https://github.com/megvii-research/megfile --skill megfile
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: megfile
Source: https://github.com/megvii-research/megfile/tree/main/scripts/skill
Command: npx skills add https://github.com/megvii-research/megfile --skill megfile

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Megfile provides a unified API and SmartPath abstractions to simplify cross-backend file access, eliminating the need to learn multiple backends and protocols.

Core Features & Use Cases

  • Unified file IO across local FS, S3/OSS, SFTP, WebDAV, HTTP, and more
  • Path utilities, metadata, and configuration via environment variables and profiles
  • CLI support and plug-in extensibility for multiple backends

Quick Start

Install megfile with the base package and optional extras; then use the SmartPath or APIs to access local and remote storage across protocols.

Frequently Asked Questions about megfile

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I access files across S3, SFTP, and local filesystems using a single Python API?

Unified file access across local and cloud backends is solved by SmartPath abstractions, enabling seamless file IO, metadata, and lifecycle operations across S3, SFTP, WebDAV, HTTP, and HDFS. You use a consistent Python API to interact with diverse protocols.

What is the best way to sync files between local storage and S3 buckets in Python?

Syncing files between local storage and S3 buckets is handled by the unified API's sync functionality. It manages cross-backend file orchestration, allowing you to synchronize paths and perform copying operations across different environments.

Can I configure S3 and SFTP backend connections using environment variables and profiles?

S3 and SFTP backend connections can be configured via environment variables and profiles. The system supports configurability through these methods, allowing you to manage credentials and settings for diverse storage backends.

Does a unified Python file API work with WebDAV and HDFS for listing and copying operations?

A unified Python file API works with WebDAV and HDFS by providing a broad API surface for existence checks, listing, and copying. It abstracts the underlying protocols to handle file IO and path utilities consistently.

How do I perform file existence checks and retrieve metadata across different cloud storage backends?

File existence checks and metadata retrieval across different cloud storage backends are performed using the unified API. It applies across local file systems and diverse backends, enabling seamless metadata queries and lifecycle operations.

Is there a CLI tool for cross-backend file operations and orchestration?

A CLI tool is available for cross-backend file operations and orchestration. It provides plug-in extensibility for multiple backends, allowing you to execute file IO and path utilities directly from the command line.