zwf
4729698049
git-subtree-dir: paste-framework git-subtree-split: 34e8684c4bc3cebbe177509f42ab4ef5b5425a7a
288 lines
11 KiB
Markdown
288 lines
11 KiB
Markdown
```markdown
|
|
┌─────────────────────────────────────────────────┐
|
|
│ │
|
|
│ ██████╗ █████╗ ███████╗████████╗███████╗ │
|
|
│ ██╔══██╗██╔══██╗██╔════╝╚══██╔══╝██╔════╝ │
|
|
│ ██████╔╝███████║███████╗ ██║ █████╗ │
|
|
│ ██╔═══╝ ██╔══██║╚════██║ ██║ ██╔══╝ │
|
|
│ ██║ ██║ ██║███████║ ██║ ███████╗ │
|
|
│ ╚═╝ ╚═╝ ╚═╝╚══════╝ ╚═╝ ╚══════╝ │
|
|
│ │
|
|
│ Python Api-first Scalable Task Engine │
|
|
│ │
|
|
└─────────────────────────────────────────────────┘
|
|
```
|
|
|
|
# PASTE — A Production-Ready Lightweight Python Framework
|
|
|
|
> A minimalist, battle-tested Python framework for building api services with **built-in RBAC, JWT, async tasks, Snowflake IDs, Swagger, and modular utilities** — zero boilerplate, maximum control. Built on top of **Tornado**.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
## ✨ Core Features
|
|
|
|
| Feature | Description |
|
|
|---------|-------------|
|
|
| 🚀 **Auto-Loading Handlers** | Define `route_pattern = "/user"` in your handler → API is automatically registered. No router config needed. |
|
|
| 🔐 **RBAC with Dynamic Rules** | Permissions stored as serialized Python objects in DB. Define complex rules as classes (e.g., `TimeBasedRule`, `IPWhitelistRule`). |
|
|
| 📜 **Swagger UI Auto-Generated** | Built-in Swagger UI at `/docs`. No YAML. Schema is inferred from handlers and decorators. |
|
|
| 🔢 **Snowflake ID Generator** | Embedded, thread-safe, no external dependency. Generates 10K+ unique IDs/sec. |
|
|
| ⚡ **Async Task Pool with Backpressure** | `run_background_task(coro)` safely manages concurrent background jobs with configurable limits. |
|
|
| 🔐 **JWT + Password Hashing** | Stateless authentication via `paste.security.token`. Passwords hashed with PBKDF2-sha256. |
|
|
| 🧰 **Modular Utility Library (`util/`)** | Clean, testable utilities: `ustr`, `udict`, `ufile`, `pagination`, `snow_id`, `encoder` — no pollution in `__init__.py`. |
|
|
| 📦 **Layered Architecture** | `core/` (config, logging, async), `db/` (engine, models), `rbac/` (roles, rules), `web/` (handlers, app), `service/` (business logic) — clean separation. |
|
|
| 🛡️ **Safe File Handling** | `sanitize_filename()` ensures cross-platform compatibility (Windows/Linux/macOS). |
|
|
| 📊 **JSON Encoder for Complex Types** | Auto-serializes `datetime`, `Decimal`, `BaseModel`, `Enum`, `numpy` — no extra config. |
|
|
| 📬 **Redis Stream Actor** | Reliable consumer-group-based message processing with auto zombie-task recovery. |
|
|
| 🧩 **ParamAware UI Modules** | Async-preload data for Tornado UIModules — solves the age-old sync-render bottleneck. |
|
|
|
|
---
|
|
|
|
## 🛠️ Quick Start
|
|
|
|
### 1. Install
|
|
|
|
```bash
|
|
git clone https://github.com/your-repo/paste.git
|
|
cd paste
|
|
pip install -e .
|
|
```
|
|
|
|
### 2. Run the example
|
|
|
|
All ready-to-run examples are located in the `examples/` directory:
|
|
|
|
```bash
|
|
cd examples/01_hello_world
|
|
python main.py
|
|
```
|
|
|
|
👉 Open: [http://localhost:9090/hello](http://localhost:9090/hello)
|
|
|
|
---
|
|
|
|
## 📚 Examples
|
|
|
|
Jump right in with these working examples:
|
|
|
|
| Example | Description | Key Features Demonstrated |
|
|
|---------|-------------|--------------------------|
|
|
| [`01_hello_world`](examples/01_hello_world/) | Minimal paste app | Auto-loading handlers, config, response helpers |
|
|
| [`02_background_task`](examples/02_background_task/) | Async background tasks | Task pool, logging, daemonized services |
|
|
| [`03_redis_stream`](examples/03_redis_stream/) | Redis Stream publisher & consumer | `StreamActor`, consumer groups, message ACK |
|
|
| [`04_tasks_service`](examples/04_tasks_service/) | Scheduled task service | `TaskService`, cron-like scheduling, PID management |
|
|
| [`05_gen_models`](examples/05_gen_models/) | Auto-generate DB models from existing tables | SQLAlchemy model generation |
|
|
|
|
Each example includes a `config.json`, a `handler.py`, and a `main.py` — ready to run.
|
|
|
|
---
|
|
|
|
## 🏗️ Framework Architecture
|
|
|
|
```
|
|
paste/ # Framework core (do NOT modify)
|
|
├── core/ # Foundation layer
|
|
│ ├── config.py # Dot-path config loader (get_config("db.engine"))
|
|
│ ├── logging.py # Logger with rotating file + console
|
|
│ └── aio_pool.py # Async task pool with backpressure
|
|
├── db/ # Database layer
|
|
│ ├── engine.py # SQLAlchemy engine factory
|
|
│ ├── redis.py # Redis connection + StreamActor
|
|
│ ├── basemodel.py # Async ORM base model
|
|
│ ├── basetable.py # Table reflection utilities
|
|
│ ├── baseadapter.py # Result-set adapter
|
|
│ └── gen_models.py # Auto-generate model classes
|
|
├── web/ # Web layer
|
|
│ ├── application.py # Tornado Application with auto-loading
|
|
│ ├── handler.py # Base RequestHandler with response helpers
|
|
│ ├── decorators.py # @route, @auth_token, @auth_permission
|
|
│ ├── swagger.py # Swagger UI auto-generation
|
|
│ ├── form.py # WTForms integration
|
|
│ ├── param_aware_loader.py # Async UIModule data preloader
|
|
│ └── websocket.py # WebSocket support
|
|
├── rbac/ # Access control layer
|
|
│ ├── rbac_user.py # User with permission inheritance
|
|
│ ├── rbac_role.py # Role management
|
|
│ ├── rbac_rule.py # Rule engine (pickle-serialized)
|
|
│ ├── rbac_item.py # Permission item hierarchy
|
|
│ ├── rbac_assignment.py # User-item assignment
|
|
│ └── rbac_permission.py # Permission query
|
|
├── security/ # Security layer
|
|
│ ├── token.py # JWT encode/decode with configurable issuer
|
|
│ └── shash.py # PBKDF2-sha256 password hashing
|
|
├── service/ # Service layer
|
|
│ ├── server.py # Server process management
|
|
│ ├── daemonize.py # Unix daemon + PID file
|
|
│ └── task_service.py # Scheduled task runner
|
|
├── chart/ # Chart generation (optional)
|
|
│ ├── bar.py / pie.py / line.py
|
|
│ └── ...
|
|
└── util/ # Utility library
|
|
├── ustr.py / udict.py # String & dict helpers
|
|
├── ufile.py # File operations with sanitized filenames
|
|
├── pagination.py # Page, Pagination, CursorPagination
|
|
├── snow_id.py # Snowflake ID generator
|
|
├── encoder.py # JSON encoder + BaseX decoder
|
|
├── pdf.py / svg.py / xlsx.py
|
|
└── ...
|
|
```
|
|
|
|
Your application code lives **entirely outside** `paste/`:
|
|
|
|
```
|
|
myapp/
|
|
├── main.py # Entry point: create Application, listen, start
|
|
├── config.json # DB, logger, RBAC, Tornado configuration
|
|
├── apps/ # Your handlers (recommended)
|
|
│ └── demo/
|
|
│ ├── __init__.py
|
|
│ ├── handler_user.py
|
|
│ └── handler_auth.py
|
|
├── models/ # Your DB models
|
|
│ └── db_models.py
|
|
└── service/ # Your background services
|
|
└── __init__.py
|
|
```
|
|
|
|
---
|
|
|
|
## 🔐 RBAC Example
|
|
|
|
```python
|
|
# handler.py
|
|
from paste.rbac.rbac_user import RbacUser
|
|
from paste.web.decorators import route, auth_token, auth_permission
|
|
from paste.web.handler import RequestHandler
|
|
|
|
@route("/admin/users")
|
|
class AdminUserHandler(RequestHandler):
|
|
@auth_token
|
|
@auth_permission
|
|
async def get(self):
|
|
users = await RbacUser.query_as_df(RbacUser().gen_query())
|
|
self.response_ok(rows=users.to_dict('records'))
|
|
```
|
|
|
|
```python
|
|
# rule.py — dynamic rule as a serialized Python class
|
|
from datetime import datetime
|
|
from paste.rbac.rbac_rule import RbacRule
|
|
|
|
class BusinessTimeRule(RbacRule):
|
|
async def run(self, **kwargs) -> bool:
|
|
hour = datetime.now().hour
|
|
return 9 <= hour < 18 # only allow during business hours
|
|
```
|
|
|
|
---
|
|
|
|
## ⚙️ Configuration
|
|
|
|
All configuration lives in a single `config.json`:
|
|
|
|
```json
|
|
{
|
|
"tornado": {
|
|
"demo": {
|
|
"autoreload": false,
|
|
"handlers_pkg": "apps.demo",
|
|
"port": 9090,
|
|
"static_path": "static",
|
|
"template_path": "templates",
|
|
"swagger_title": "DemoAPI",
|
|
"swagger_description": "Demo API",
|
|
"swagger_api_version": "1.0.1"
|
|
}
|
|
},
|
|
"db": {
|
|
"engine": {
|
|
"engine": "mysql+pymysql://user:pass@localhost:3306/mydb",
|
|
"async_engine": "mysql+aiomysql://user:pass@localhost:3306/mydb",
|
|
"engine_option": {
|
|
"echo": false,
|
|
"pool_size": 10
|
|
}
|
|
}
|
|
},
|
|
"redis": {
|
|
"connection": "redis://localhost:6379/0",
|
|
"streams": {
|
|
"user_event": {
|
|
"group": "user_event_group",
|
|
"consumer": "processor_01"
|
|
}
|
|
}
|
|
},
|
|
"rbac": {
|
|
"user_class": "myapp.models.MyRbacUser",
|
|
"table": {
|
|
"rule": "rbac_rule",
|
|
"user": "rbac_user",
|
|
"item": "rbac_item",
|
|
"assignment": "rbac_assignment",
|
|
"item_child": "rbac_item_child"
|
|
}
|
|
},
|
|
"logger": {
|
|
"default": {
|
|
"basic": {
|
|
"filename": "logs/app.log",
|
|
"format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s",
|
|
"level": 20
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Access any config value with dot-path:
|
|
|
|
```python
|
|
from paste.core import config
|
|
engine_url = config.get_config("db.engine.engine")
|
|
port = config.get_config("tornado.demo.port", 9000)
|
|
```
|
|
|
|
---
|
|
|
|
## 📣 License
|
|
|
|
MIT © 2025 [Wayne Zhang / Organization]
|
|
|
|
---
|
|
|
|
## 🌍 Contributing
|
|
|
|
Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on:
|
|
- Code style (Black + flake8 + mypy)
|
|
- How to submit issues and pull requests
|
|
|
|
---
|
|
|
|
## 🧪 Running Tests
|
|
|
|
```bash
|
|
pytest # Run all tests
|
|
pytest tests/unit # Run unit tests only (no external services)
|
|
pytest tests/integration # Run integration tests (require DB/Redis)
|
|
pytest --cov=paste # Run with coverage report
|
|
```
|
|
|
|
---
|
|
|
|
> **💡 Why paste?**
|
|
>
|
|
> Most frameworks make you write boilerplate.
|
|
> `paste` makes you **stop writing boilerplate** — and **start building features**.
|
|
>
|
|
> - ✅ No `@app.get("/user")` decorators — just `route_pattern = "/user"`
|
|
> - ✅ No YAML for Swagger — schema inferred automatically
|
|
> - ✅ No Redis for IDs — Snowflake built-in
|
|
> - ✅ No Celery for tasks — async pool with backpressure
|
|
> - ✅ No config files in 5 places — one `config.json`, one `get_config("db.engine")`
|
|
```
|