Squashed 'paste-framework/' content from commit 34e8684
git-subtree-dir: paste-framework git-subtree-split: 34e8684c4bc3cebbe177509f42ab4ef5b5425a7a
This commit is contained in:
zwf
@@ -0,0 +1,287 @@
|
||||
```markdown
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ │
|
||||
│ ██████╗ █████╗ ███████╗████████╗███████╗ │
|
||||
│ ██╔══██╗██╔══██╗██╔════╝╚══██╔══╝██╔════╝ │
|
||||
│ ██████╔╝███████║███████╗ ██║ █████╗ │
|
||||
│ ██╔═══╝ ██╔══██║╚════██║ ██║ ██╔══╝ │
|
||||
│ ██║ ██║ ██║███████║ ██║ ███████╗ │
|
||||
│ ╚═╝ ╚═╝ ╚═╝╚══════╝ ╚═╝ ╚══════╝ │
|
||||
│ │
|
||||
│ Python Api-first Scalable Task Engine │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
# PASTE — A Production-Ready Lightweight Python Framework
|
||||
|
||||
> A minimalist, battle-tested Python framework for building api services with **built-in RBAC, JWT, async tasks, Snowflake IDs, Swagger, and modular utilities** — zero boilerplate, maximum control. Built on top of **Tornado**.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## ✨ Core Features
|
||||
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| 🚀 **Auto-Loading Handlers** | Define `route_pattern = "/user"` in your handler → API is automatically registered. No router config needed. |
|
||||
| 🔐 **RBAC with Dynamic Rules** | Permissions stored as serialized Python objects in DB. Define complex rules as classes (e.g., `TimeBasedRule`, `IPWhitelistRule`). |
|
||||
| 📜 **Swagger UI Auto-Generated** | Built-in Swagger UI at `/docs`. No YAML. Schema is inferred from handlers and decorators. |
|
||||
| 🔢 **Snowflake ID Generator** | Embedded, thread-safe, no external dependency. Generates 10K+ unique IDs/sec. |
|
||||
| ⚡ **Async Task Pool with Backpressure** | `run_background_task(coro)` safely manages concurrent background jobs with configurable limits. |
|
||||
| 🔐 **JWT + Password Hashing** | Stateless authentication via `paste.security.token`. Passwords hashed with PBKDF2-sha256. |
|
||||
| 🧰 **Modular Utility Library (`util/`)** | Clean, testable utilities: `ustr`, `udict`, `ufile`, `pagination`, `snow_id`, `encoder` — no pollution in `__init__.py`. |
|
||||
| 📦 **Layered Architecture** | `core/` (config, logging, async), `db/` (engine, models), `rbac/` (roles, rules), `web/` (handlers, app), `service/` (business logic) — clean separation. |
|
||||
| 🛡️ **Safe File Handling** | `sanitize_filename()` ensures cross-platform compatibility (Windows/Linux/macOS). |
|
||||
| 📊 **JSON Encoder for Complex Types** | Auto-serializes `datetime`, `Decimal`, `BaseModel`, `Enum`, `numpy` — no extra config. |
|
||||
| 📬 **Redis Stream Actor** | Reliable consumer-group-based message processing with auto zombie-task recovery. |
|
||||
| 🧩 **ParamAware UI Modules** | Async-preload data for Tornado UIModules — solves the age-old sync-render bottleneck. |
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Quick Start
|
||||
|
||||
### 1. Install
|
||||
|
||||
```bash
|
||||
git clone https://github.com/your-repo/paste.git
|
||||
cd paste
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
### 2. Run the example
|
||||
|
||||
All ready-to-run examples are located in the `examples/` directory:
|
||||
|
||||
```bash
|
||||
cd examples/01_hello_world
|
||||
python main.py
|
||||
```
|
||||
|
||||
👉 Open: [http://localhost:9090/hello](http://localhost:9090/hello)
|
||||
|
||||
---
|
||||
|
||||
## 📚 Examples
|
||||
|
||||
Jump right in with these working examples:
|
||||
|
||||
| Example | Description | Key Features Demonstrated |
|
||||
|---------|-------------|--------------------------|
|
||||
| [`01_hello_world`](examples/01_hello_world/) | Minimal paste app | Auto-loading handlers, config, response helpers |
|
||||
| [`02_background_task`](examples/02_background_task/) | Async background tasks | Task pool, logging, daemonized services |
|
||||
| [`03_redis_stream`](examples/03_redis_stream/) | Redis Stream publisher & consumer | `StreamActor`, consumer groups, message ACK |
|
||||
| [`04_tasks_service`](examples/04_tasks_service/) | Scheduled task service | `TaskService`, cron-like scheduling, PID management |
|
||||
| [`05_gen_models`](examples/05_gen_models/) | Auto-generate DB models from existing tables | SQLAlchemy model generation |
|
||||
|
||||
Each example includes a `config.json`, a `handler.py`, and a `main.py` — ready to run.
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Framework Architecture
|
||||
|
||||
```
|
||||
paste/ # Framework core (do NOT modify)
|
||||
├── core/ # Foundation layer
|
||||
│ ├── config.py # Dot-path config loader (get_config("db.engine"))
|
||||
│ ├── logging.py # Logger with rotating file + console
|
||||
│ └── aio_pool.py # Async task pool with backpressure
|
||||
├── db/ # Database layer
|
||||
│ ├── engine.py # SQLAlchemy engine factory
|
||||
│ ├── redis.py # Redis connection + StreamActor
|
||||
│ ├── basemodel.py # Async ORM base model
|
||||
│ ├── basetable.py # Table reflection utilities
|
||||
│ ├── baseadapter.py # Result-set adapter
|
||||
│ └── gen_models.py # Auto-generate model classes
|
||||
├── web/ # Web layer
|
||||
│ ├── application.py # Tornado Application with auto-loading
|
||||
│ ├── handler.py # Base RequestHandler with response helpers
|
||||
│ ├── decorators.py # @route, @auth_token, @auth_permission
|
||||
│ ├── swagger.py # Swagger UI auto-generation
|
||||
│ ├── form.py # WTForms integration
|
||||
│ ├── param_aware_loader.py # Async UIModule data preloader
|
||||
│ └── websocket.py # WebSocket support
|
||||
├── rbac/ # Access control layer
|
||||
│ ├── rbac_user.py # User with permission inheritance
|
||||
│ ├── rbac_role.py # Role management
|
||||
│ ├── rbac_rule.py # Rule engine (pickle-serialized)
|
||||
│ ├── rbac_item.py # Permission item hierarchy
|
||||
│ ├── rbac_assignment.py # User-item assignment
|
||||
│ └── rbac_permission.py # Permission query
|
||||
├── security/ # Security layer
|
||||
│ ├── token.py # JWT encode/decode with configurable issuer
|
||||
│ └── shash.py # PBKDF2-sha256 password hashing
|
||||
├── service/ # Service layer
|
||||
│ ├── server.py # Server process management
|
||||
│ ├── daemonize.py # Unix daemon + PID file
|
||||
│ └── task_service.py # Scheduled task runner
|
||||
├── chart/ # Chart generation (optional)
|
||||
│ ├── bar.py / pie.py / line.py
|
||||
│ └── ...
|
||||
└── util/ # Utility library
|
||||
├── ustr.py / udict.py # String & dict helpers
|
||||
├── ufile.py # File operations with sanitized filenames
|
||||
├── pagination.py # Page, Pagination, CursorPagination
|
||||
├── snow_id.py # Snowflake ID generator
|
||||
├── encoder.py # JSON encoder + BaseX decoder
|
||||
├── pdf.py / svg.py / xlsx.py
|
||||
└── ...
|
||||
```
|
||||
|
||||
Your application code lives **entirely outside** `paste/`:
|
||||
|
||||
```
|
||||
myapp/
|
||||
├── main.py # Entry point: create Application, listen, start
|
||||
├── config.json # DB, logger, RBAC, Tornado configuration
|
||||
├── apps/ # Your handlers (recommended)
|
||||
│ └── demo/
|
||||
│ ├── __init__.py
|
||||
│ ├── handler_user.py
|
||||
│ └── handler_auth.py
|
||||
├── models/ # Your DB models
|
||||
│ └── db_models.py
|
||||
└── service/ # Your background services
|
||||
└── __init__.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 RBAC Example
|
||||
|
||||
```python
|
||||
# handler.py
|
||||
from paste.rbac.rbac_user import RbacUser
|
||||
from paste.web.decorators import route, auth_token, auth_permission
|
||||
from paste.web.handler import RequestHandler
|
||||
|
||||
@route("/admin/users")
|
||||
class AdminUserHandler(RequestHandler):
|
||||
@auth_token
|
||||
@auth_permission
|
||||
async def get(self):
|
||||
users = await RbacUser.query_as_df(RbacUser().gen_query())
|
||||
self.response_ok(rows=users.to_dict('records'))
|
||||
```
|
||||
|
||||
```python
|
||||
# rule.py — dynamic rule as a serialized Python class
|
||||
from datetime import datetime
|
||||
from paste.rbac.rbac_rule import RbacRule
|
||||
|
||||
class BusinessTimeRule(RbacRule):
|
||||
async def run(self, **kwargs) -> bool:
|
||||
hour = datetime.now().hour
|
||||
return 9 <= hour < 18 # only allow during business hours
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
All configuration lives in a single `config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"tornado": {
|
||||
"demo": {
|
||||
"autoreload": false,
|
||||
"handlers_pkg": "apps.demo",
|
||||
"port": 9090,
|
||||
"static_path": "static",
|
||||
"template_path": "templates",
|
||||
"swagger_title": "DemoAPI",
|
||||
"swagger_description": "Demo API",
|
||||
"swagger_api_version": "1.0.1"
|
||||
}
|
||||
},
|
||||
"db": {
|
||||
"engine": {
|
||||
"engine": "mysql+pymysql://user:pass@localhost:3306/mydb",
|
||||
"async_engine": "mysql+aiomysql://user:pass@localhost:3306/mydb",
|
||||
"engine_option": {
|
||||
"echo": false,
|
||||
"pool_size": 10
|
||||
}
|
||||
}
|
||||
},
|
||||
"redis": {
|
||||
"connection": "redis://localhost:6379/0",
|
||||
"streams": {
|
||||
"user_event": {
|
||||
"group": "user_event_group",
|
||||
"consumer": "processor_01"
|
||||
}
|
||||
}
|
||||
},
|
||||
"rbac": {
|
||||
"user_class": "myapp.models.MyRbacUser",
|
||||
"table": {
|
||||
"rule": "rbac_rule",
|
||||
"user": "rbac_user",
|
||||
"item": "rbac_item",
|
||||
"assignment": "rbac_assignment",
|
||||
"item_child": "rbac_item_child"
|
||||
}
|
||||
},
|
||||
"logger": {
|
||||
"default": {
|
||||
"basic": {
|
||||
"filename": "logs/app.log",
|
||||
"format": "%(asctime)s - %(name)s - %(levelname)s - %(message)s",
|
||||
"level": 20
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Access any config value with dot-path:
|
||||
|
||||
```python
|
||||
from paste.core import config
|
||||
engine_url = config.get_config("db.engine.engine")
|
||||
port = config.get_config("tornado.demo.port", 9000)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📣 License
|
||||
|
||||
MIT © 2025 [Wayne Zhang / Organization]
|
||||
|
||||
---
|
||||
|
||||
## 🌍 Contributing
|
||||
|
||||
Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on:
|
||||
- Code style (Black + flake8 + mypy)
|
||||
- How to submit issues and pull requests
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Running Tests
|
||||
|
||||
```bash
|
||||
pytest # Run all tests
|
||||
pytest tests/unit # Run unit tests only (no external services)
|
||||
pytest tests/integration # Run integration tests (require DB/Redis)
|
||||
pytest --cov=paste # Run with coverage report
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
> **💡 Why paste?**
|
||||
>
|
||||
> Most frameworks make you write boilerplate.
|
||||
> `paste` makes you **stop writing boilerplate** — and **start building features**.
|
||||
>
|
||||
> - ✅ No `@app.get("/user")` decorators — just `route_pattern = "/user"`
|
||||
> - ✅ No YAML for Swagger — schema inferred automatically
|
||||
> - ✅ No Redis for IDs — Snowflake built-in
|
||||
> - ✅ No Celery for tasks — async pool with backpressure
|
||||
> - ✅ No config files in 5 places — one `config.json`, one `get_config("db.engine")`
|
||||
```
|
||||
Reference in New Issue
Block a user