初始化项目

This commit is contained in:
zwf
2026-06-02 17:46:38 +08:00
commit 646a4d02c0
240 changed files with 33662 additions and 0 deletions
+8
View File
@@ -0,0 +1,8 @@
# 默认忽略的文件
/shelf/
/workspace.xml
# 基于编辑器的 HTTP 客户端请求
/httpRequests/
# Datasource local storage ignored files
/dataSources/
/dataSources.local.xml
Generated
+4
View File
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="VcsDirectoryMappings" defaultProject="true" />
</project>
+130
View File
@@ -0,0 +1,130 @@
# 数字化三方系统集成(D3I
## 政务服务单位统一工单中枢平台
---
### 项目概述
数字化三方系统集成(Digital 3-Directional Integration, D3I)平台,是面向城市公共服务单位的**统一工单中枢系统**,旨在彻底解决一线工作人员“**多系统登录、多平台响应、重复录入、信息割裂**”的痛点。
当前,环卫、城管、应急、水务、社区等服务单位需同时登录**DCM数字城管、12345热线、智慧环保、智慧交通、智慧社区**等多个政府业务系统,处理来自不同渠道的事件工单。这不仅导致**工作效率低下**,更因系统间数据不通,造成**任务漏办、重复派单、反馈滞后**。
本系统以“**一屏统揽、一单通办、一键归集**”为目标,**不新增上报入口,不改变原有业务流程**,而是作为**政府各业务系统与服务单位OA系统之间的智能数据聚合层**,将分散在各平台的事件任务自动汇聚、标准化、结构化,并推送至服务单位统一的工作台(对接OA),实现:
-**一个系统**(OA)处理**所有来源**的事件
-**一次登录**查看**全部任务**,无需切换系统
-**自动提取**位置、附件、流程、优先级,**无需人工抄录**
-**统一归档**,形成完整处置档案,支撑绩效考核
> 🎯 **核心价值**:**让一线人员从“系统操作员”回归“问题解决者”**
---
### 核心功能模块
| 模块 | 功能描述 |
|------|----------|
| **统一身份中心** | 与服务单位OA系统对接,支持单点登录(SSO),自动映射用户角色(如环卫工、网格员),无需重复登录多个政府系统 |
| **事件汇聚引擎** | 定时从DCM、省12345、智慧环保、智慧交通等政府系统拉取事件数据,自动清洗、去重、标准化,统一为结构化工单(含位置、类型、附件、来源系统、时间戳) |
| **智能归集中心** | 基于事件类型、责任单位、地理位置,自动匹配服务单位内部责任科室/人员,生成“**OA工单**”并推送至其工作台,支持微信/短信提醒 |
| **协同处置工作台** | 在服务单位OA系统内展示统一工单列表,支持查看原始来源、处理流程、附件、历史记录,支持内部流转、备注、状态更新,**无需跳转至原系统** |
| **数据驾驶舱** | 为管理层提供服务单位任务量分布、响应时效、重复派单率、任务闭环率等指标,辅助优化资源配置 |
| **反馈闭环机制** | 服务单位在OA内完成处置后,自动回传处理结果至原系统(如DCM),完成闭环,避免“办完不反馈” |
---
### 技术架构(Mermaid 架构图)
```mermaid
graph TD
A[DCM系统] -->|拉取| E[事件汇聚引擎]
B[省12345平台] -->|拉取| E
C[智慧环保系统] -->|拉取| E
D[智慧交通系统] -->|拉取| E
F[智慧社区系统] -->|拉取| E
E --> G[智能归集中心<br>去重+标准化+匹配责任单位]
G --> H[服务单位OA系统<br>统一工单入口]
H --> I[一线人员<br>处理任务、上传结果]
I --> J[自动回传<br>至原系统(DCM/12345等)]
G --> K[数据驾驶舱<br>任务量/时效/闭环率分析]
style A fill:#e6f7ff,stroke:#1890ff
style B fill:#e6f7ff,stroke:#1890ff
style C fill:#e6f7ff,stroke:#1890ff
style D fill:#e6f7ff,stroke:#1890ff
style F fill:#e6f7ff,stroke:#1890ff
style E fill:#fff2e8,stroke:#d4380d
style G fill:#f9f0ff,stroke:#722ed1
style H fill:#f6ffed,stroke:#52c41a
style I fill:#fff7e6,stroke:#fa8c16
style J fill:#f6ffed,stroke:#52c41a
style K fill:#fff0f6,stroke:#eb2f96
```
> **图注**:本系统**不面向市民**,也不替代政府原有系统。它是一个**隐身在后台的“数据翻译器”和“任务调度器”**,让服务单位只需关注一个系统——他们的OA。
---
### 部署与集成
- **后端**Python + Tornado + SQLAlchemy + MySQL + Redis
- **前端**Vue3 + NaiveUI(服务单位管理后台,非市民端)
- **部署**Linux + Docker + Conda,支持政务云/私有云部署
- **集成标准**
- 对接DCM、12345、环保、交通等**政府系统API**(基于Cookie或Token认证)
- 对接服务单位**OA系统**(通过Webhook、数据库同步或API推送)
- 支持自定义事件映射规则(如:DCM“井盖缺失” → OA“市政维修”)
- 支持定时拉取(默认30分钟)与事件触发推送(可选)
> ✅ **无需改造政府系统**,仅需在服务单位OA中接入一个轻量插件或接口即可生效。
---
### 使用场景示例
1. **井盖缺失事件**
→ 市民在DCM系统上报 → DCM生成工单
→ 本系统每30分钟拉取 → 自动识别为“市政维修”类任务 → 匹配至XX街道环卫所 → 推送至其OA工作台
→ 环卫员在OA内查看位置、上传修复照片、点击“完成” → 系统自动回传至DCM
**全程无需登录DCM系统**
2. **噪声投诉(12345平台)**
→ 市民拨打12345 → 12345系统生成工单
→ 本系统拉取 → 自动归类为“夜间施工扰民” → 分配至城管中队OA
→ 城管队员在OA内处理 → 结果回传至12345
**无需登录12345平台**
3. **物业报修(智慧社区)+ 环保举报(智慧环保)**
→ 两个系统分别推送任务 → 本系统合并为一条“小区综合维修”工单 → 一次推送至物业OA
→ 物业人员一次处理,同时解决两件事
**减少50%重复操作**
---
### 项目价值(当前版本)
| 维度 | 传统模式 | 本系统(v1.0) |
|------|----------|----------------|
| 系统登录次数/日 | 5–8次 | **1次(仅OA** |
| 任务录入耗时 | 平均8–12分钟/单 | **≤2分钟/单(自动填充)** |
| 任务漏办率 | 1520% | **<3%** |
| 多系统数据一致性 | 无 | **100%统一** |
| 一线人员满意度 | 58% | **目标提升至90%+** |
> ✅ **核心价值**:**每天节省2小时操作时间,让一线人员真正“把时间用在解决问题上”**
---
### 扩展
本系统为**政务数字化“减负工程”标杆项目**:
- 文档:`docs/` 目录下含 API 手册、对接指南、事件映射规则模板
- 模块可插拔:支持快速接入新政府系统(只需配置采集规则)
- 支持“**反向推送**”:服务单位在OA内发起的工单,也可反向推送至政府系统(如“申请物资”)
- 未来可升级为**城市公共服务数字员工平台**,支持AI自动分类、智能推荐处理人
© 2026 数字化三方系统集成项目组 · 智慧城市治理创新实验室
+32
View File
@@ -0,0 +1,32 @@
"""
全局参数。
"""
from paste.core import config
__package_name__ = "D3I"
__version__ = config.get_config('version')
__author__ = "苏州皓楷信息技术有限公司"
__email__ = "waynezwf@qq.com"
def get_version(ver = __version__):
"""
系统版本。
:param ver:
:return:
"""
return f"{__package_name__} version: V{ver}, written by {__author__}."
def get_active_env():
"""
取得激活的环境。
:return:
"""
return config.get_config('active_env')
Binary file not shown.
Binary file not shown.
View File
Binary file not shown.
+8
View File
@@ -0,0 +1,8 @@
"""
数字城管接口。
"""
ApiPrefix = "/system/digital/city/management"
"""
API 前缀。
"""
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+102
View File
@@ -0,0 +1,102 @@
"""
接受OA请求,操作数字城管工单延期
"""
import logging
from typing import Optional
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_push_apply_postpone
from models.dcm_apply_delay import DcmApplyPostpone
from models.dcm_task import DcmTask
from paste.core import aio_pool
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/applyDelay')
class ApplyPostponeHandler(AppHandler):
"""
申请延期接口。
对接数字城管系统的申请延期接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dcm_task: Optional[DcmTask] = None
self.dcm_apply_postpone: Optional[DcmApplyPostpone] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
DcmApplyPostpone.flow_token.key: kwargs.get('flowToken', ''),
DcmApplyPostpone.dcm_task_id.key: kwargs.get('gdId', ''),
DcmApplyPostpone.task_number.key: kwargs.get('taskNumber', ''),
DcmApplyPostpone.apply_act_id.key: self.dcm_task.act_id,
DcmApplyPostpone.reply_part_id.key: kwargs.get('replyPartID', 39),
DcmApplyPostpone.ard_level.key: kwargs.get('ardLevel', 0),
DcmApplyPostpone.ard_type_id.key: kwargs.get('ardTypeId', 12),
DcmApplyPostpone.apply_memo.key: kwargs.get('opinion', ''),
DcmApplyPostpone.apply_type.key: kwargs.get('applyType', '延期'),
DcmApplyPostpone.attachments.key: kwargs.get('attachments', ''),
DcmApplyPostpone.delay_multiple.key: kwargs.get('delayMultiple', 2),
DcmApplyPostpone.time_num.key: kwargs.get('timeNum', 48),
DcmApplyPostpone.time_unit.key: kwargs.get('timeUnit', 4),
DcmApplyPostpone.postpone_date.key: kwargs.get('postponeDate', ''),
}
async def apply_postpone(self, **kwargs) -> dict:
# 必填参数校验
required_keys = [
'gdId', 'taskNumber', 'applyType', 'opinion', 'delayMultiple', 'flowToken'
]
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
if kwargs.get('delayMultiple') not in (1, 2, '1', '2'):
raise ValueError('延期倍数只能为1或2')
# 读取待办任务对象
dcm_task_id = kwargs.get('gdId', '')
self.dcm_task = await DcmTask.async_find_by_id(dcm_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.dcm_apply_postpone = DcmApplyPostpone().copy_from_dict(params)
self.dcm_apply_postpone.status = 0
await self.dcm_apply_postpone.async_save()
# 后台执行提交申请延期请求到数字城管
await aio_pool.run_background_task(
dcm_push_apply_postpone.push_apply_postpone(self.dcm_apply_postpone, self.dcm_task)
)
return {
'msg': '申请延期成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 申请延期接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.apply_postpone(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+97
View File
@@ -0,0 +1,97 @@
"""
接受OA请求,操作数字城管的申请回退接口
"""
import logging
from typing import Optional
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_push_apply_rollback
from models.dcm_apply_rollback import DcmApplyRollback
from models.dcm_task import DcmTask
from paste.core import aio_pool
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/applyRollback')
class ApplyRollbackHandler(AppHandler):
"""
申请回退接口。
对接数字城管系统的申请回退接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dcm_task: Optional[DcmTask] = None
self.dcm_apply_rollback: Optional[DcmApplyRollback] = None
def _extract_params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
DcmApplyRollback.flow_token.key: kwargs.get('flowToken', ''),
DcmApplyRollback.dcm_task_id.key: kwargs.get('gdId', ''),
DcmApplyRollback.act_id.key: self.dcm_task.act_id,
DcmApplyRollback.task_number.key: kwargs.get('taskNumber', ''),
DcmApplyRollback.reply_part_id.key: kwargs.get('replyPartID', 39),
DcmApplyRollback.ard_level.key: kwargs.get('ardLevel', 0),
DcmApplyRollback.ard_type_id.key: 18 if kwargs.get('applyType', '拒签') == '拒签' else 62,
DcmApplyRollback.opinion.key: kwargs.get('opinion', ''),
DcmApplyRollback.apply_type.key: kwargs.get('applyType', '拒签'),
DcmApplyRollback.trans_info.key: kwargs.get('transInfo', '52,254,0'),
DcmApplyRollback.attachments.key: kwargs.get('attachments', '')
}
async def apply_rollback(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId', 'taskNumber', 'opinion', 'applyType', 'flowToken']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
if kwargs['applyType'] not in ('拒签', '处置阶段照片未公开'):
raise ValueError('申请类型只能为拒签或处置阶段照片未公开')
# 读取待办任务对象
dcm_task_id = kwargs.get('gdId', '')
self.dcm_task = await DcmTask.async_find_by_id(dcm_task_id)
# 保存请求数据
params = self._extract_params_for_db(**kwargs)
self.dcm_apply_rollback = DcmApplyRollback().copy_from_dict(params)
self.dcm_apply_rollback.status = 0
await self.dcm_apply_rollback.async_save()
# 后台执行提交申请回退请求到数字城管
await aio_pool.run_background_task(
dcm_push_apply_rollback.push_apply_rollback(self.dcm_apply_rollback, self.dcm_task)
)
return {
'msg': '申请回退成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 申请回退接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.apply_rollback(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+96
View File
@@ -0,0 +1,96 @@
"""
接受OA请求,操作数字城管的工单批转接口
"""
import logging
from typing import Optional
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_push_dispose
from models.dcm_dispose import DcmDispose
from models.dcm_task import DcmTask
from paste.core import aio_pool
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/transfer')
class DisposeHandler(AppHandler):
"""
批转接口。
对接数字城管系统的批转接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dcm_task: Optional[DcmTask] = None
self.dcm_dispose: Optional[DcmDispose] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
DcmDispose.flow_token.key: kwargs.get('flowToken', ''),
DcmDispose.dcm_task_id.key: kwargs.get('gdId', ''),
DcmDispose.act_id.key: self.dcm_task.act_id,
DcmDispose.task_number.key: kwargs.get('taskNumber', ''),
DcmDispose.opinion.key: kwargs.get('opinion', ''),
DcmDispose.attachments.key: kwargs.get('attachments', ''),
DcmDispose.send_message.key: kwargs.get('sendMessage', '1'),
DcmDispose.trans_info.key: '52,254,0,0',
DcmDispose.add_num.key: kwargs.get('addNum', '0'),
DcmDispose.task_list_id.key: kwargs.get('taskListId', '600058'),
DcmDispose.undertake_user_name.key: kwargs.get('undertakeUserName', ''),
DcmDispose.undertake_phone.key: kwargs.get('undertakePhone', '')
}
async def dispose(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId', 'taskNumber', 'opinion', 'attachments', 'flowToken']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
dcm_task_id = kwargs.get('gdId', '')
self.dcm_task = await DcmTask.async_find_by_id(dcm_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.dcm_dispose = DcmDispose().copy_from_dict(params)
self.dcm_dispose.status = 0
await self.dcm_dispose.async_save()
# 后台执行提交批转请求到数字城管
await aio_pool.run_background_task(
dcm_push_dispose.push_dispose(self.dcm_dispose, self.dcm_task)
)
return {
'msg': '批转成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 批转接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.dispose(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+58
View File
@@ -0,0 +1,58 @@
"""
接受OA请求,读取数字城管的是否允许申请延期。
"""
import logging
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_scrape_allow_postpone
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/fetchAllowPostpone')
class AllowPostponeHandler(AppHandler):
"""
获取是否允许申请延期接口。
对接数字城管系统的获取是否允许申请延期接口,用于判断工单有哪些是否允许申请延期。
"""
async def fetch_allow_postpone(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
dcm_task_id = kwargs.get('gdId', '')
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
success, message = await dcm_scrape_allow_postpone.fetch_allow_postpone(dcm_task)
return {
'success': success,
'msg': message,
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 获取是否允许申请延期接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.fetch_allow_postpone(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+59
View File
@@ -0,0 +1,59 @@
"""
接受OA请求,读取数字城管的便民表单。
"""
import logging
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_scrape_conv_dispose
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/fetchDisposeForm')
class FetchConvenientFormHandler(AppHandler):
"""
获取便民表单接口。
对接数字城管系统的获取便民表单接口,用于判断工单有哪些便民表单。
"""
async def fetch_form(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId', 'formId']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
dcm_task_id = kwargs.get('gdId', '')
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
form = await dcm_scrape_conv_dispose.fetch_form(dcm_task)
return {
'msg': '获取便民批转表单成功.',
'form': form,
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 获取便民表单接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.fetch_form(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+58
View File
@@ -0,0 +1,58 @@
"""
接受OA请求,读取数字城管的可用操作。
"""
import logging
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_scrape_operation
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/fetchOperation')
class FetchOperationHandler(AppHandler):
"""
获取可用操作接口。
对接数字城管系统的获取可用操作接口,用于判断工单有哪些可用操作。
"""
async def fetch_operations(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
dcm_task_id = kwargs.get('gdId', '')
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
operations = await dcm_scrape_operation.fetch_operation(dcm_task)
return {
'msg': '获取可用操作成功.',
'operations': operations,
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 获取可用操作接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.fetch_operations(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+59
View File
@@ -0,0 +1,59 @@
"""
接受OA请求,读取数字城管的便民表单。
"""
import logging
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_scrape_conv_rollback
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/fetchRollbackForm')
class FetchRollbackFormHandler(AppHandler):
"""
获取便民表单接口。
对接数字城管系统的获取便民表单接口,用于判断工单有哪些便民表单。
"""
async def fetch_form(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId', 'formId']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
dcm_task_id = kwargs.get('gdId', '')
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
form = await dcm_scrape_conv_rollback.fetch_form(dcm_task)
return {
'msg': '获取便民回退表单成功.',
'form': form,
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 获取便民表单接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.fetch_form(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+98
View File
@@ -0,0 +1,98 @@
"""
接受OA请求,操作数字城管的回退接口
"""
import logging
from typing import Optional
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_push_rollback
from models.dcm_rollback import DcmRollback
from models.dcm_task import DcmTask
from paste.core import aio_pool
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/rollback')
class RollbackHandler(AppHandler):
"""
回退接口。
对接数字城管系统的回退接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dcm_task: Optional[DcmTask] = None
self.dcm_rollback: Optional[DcmRollback] = None
def _extract_params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
DcmRollback.flow_token.key: kwargs.get('flowToken', ''),
DcmRollback.dcm_task_id.key: kwargs.get('gdId', ''),
DcmRollback.act_id.key: self.dcm_task.act_id,
DcmRollback.task_number.key: kwargs.get('taskNumber', ''),
DcmRollback.opinion.key: kwargs.get('opinion', ''),
DcmRollback.attachments.key: kwargs.get('attachments', ''),
DcmRollback.send_message.key: kwargs.get('sendMessage', '1'),
DcmRollback.trans_info.key: kwargs.get('transInfo', '50,254,0'),
DcmRollback.save_old_act_flag.key: kwargs.get('saveOldActFlag', False),
DcmRollback.rollback_reason_id.key: kwargs.get('rollbackReasonId', -1),
DcmRollback.not_assigned.key: kwargs.get('notAssigned', '0'),
DcmRollback.not_assigned_reason.key: kwargs.get('notAssignedReason', ''),
DcmRollback.undertake_user_name.key: kwargs.get('undertakeUserName', ''),
DcmRollback.undertake_phone.key: kwargs.get('undertakePhone', '')
}
async def rollback(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId', 'taskNumber', 'opinion', 'flowToken']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
dcm_task_id = kwargs.get('gdId', '')
self.dcm_task = await DcmTask.async_find_by_id(dcm_task_id)
# 保存请求数据
params = self._extract_params_for_db(**kwargs)
self.dcm_rollback = DcmRollback().copy_from_dict(params)
self.dcm_rollback.status = 0
await self.dcm_rollback.async_save()
# 后台执行提交回退请求到数字城管
await aio_pool.run_background_task(
dcm_push_rollback.push_rollback(self.dcm_rollback, self.dcm_task)
)
return {
'msg': '回退成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 回退接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.rollback(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+91
View File
@@ -0,0 +1,91 @@
"""
接受OA请求,操作数字城管的阶段回复接口
"""
import logging
from typing import Optional
from apps.api import dcm
from apps.app_handler import AppHandler
from dock.dcm import dcm_push_stage_reply
from models.dcm_stage_reply import DcmStageReply
from models.dcm_task import DcmTask
from paste.core import aio_pool
from paste.core.logging import echo_log
from paste.web.decorators import route
@route(f'{dcm.ApiPrefix}/stageReply')
class StageReplyHandler(AppHandler):
"""
阶段回复接口。
对接数字城管系统的阶段回复接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.dcm_task: Optional[DcmTask] = None
self.dcm_stage_reply: Optional[DcmStageReply] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
DcmStageReply.flow_token.key: kwargs.get('flowToken', ''),
DcmStageReply.dcm_task_id.key: kwargs.get('gdId', ''),
DcmStageReply.rec_id.key: self.dcm_task.rec_id,
DcmStageReply.act_id.key: self.dcm_task.act_id,
DcmStageReply.content.key: kwargs.get('content', ''),
DcmStageReply.task_number.key: kwargs.get('taskNumber', ''),
DcmStageReply.item_type.key: kwargs.get('itemType', 'stage_reply'),
}
async def stage_reply(self, **kwargs) -> dict:
# 必填参数校验
required_keys = ['gdId', 'taskNumber', 'content', 'flowToken']
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
dcm_task_id = kwargs.get('gdId', '')
self.dcm_task = await DcmTask.async_find_by_id(dcm_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.dcm_stage_reply = DcmStageReply().copy_from_dict(params)
self.dcm_stage_reply.status = 1
await self.dcm_stage_reply.async_save()
# 后台执行提交阶段回复请求到数字城管
await aio_pool.run_background_task(
dcm_push_stage_reply.push_stage_reply(self.dcm_stage_reply, self.dcm_task)
)
return {
'msg': '阶段回复成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 阶段回复接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.stage_reply(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
View File
+8
View File
@@ -0,0 +1,8 @@
"""
省12345接口。
"""
ApiPrefix = "/system"
"""
API 前缀。
"""
Binary file not shown.
+101
View File
@@ -0,0 +1,101 @@
from typing import Optional
import logging
from apps.api import govs
from apps.app_handler import AppHandler
from paste.web.decorators import route
from paste.core import aio_pool
from paste.core.logging import echo_log
from dock.govs import govs_create_order_delay
from models.govs_order_master import GovsOrderMaster
from models.govs_create_delay import GovsApplicationForDelay
@route(f'{govs.ApiPrefix}/application-for-delay-formal/create')
class CreateDelayHandler(AppHandler):
"""
申请延期接口。
对接省12345的申请延期接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.govs_order: Optional[GovsOrderMaster] = None
self.govs_delay: Optional[GovsApplicationForDelay] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
GovsApplicationForDelay.master_id.key: kwargs.get('gdId', ''),
GovsApplicationForDelay.gd_id.key: kwargs.get('gdId', ''),
GovsApplicationForDelay.finally_time_after_approve.key: kwargs.get('finallyTimeAfterApprove', ''),
GovsApplicationForDelay.finally_time_before_approve.key: kwargs.get('finallyTimeBeforeApprove', ''),
GovsApplicationForDelay.request_delay.key: kwargs.get('requestDelay', ''),
GovsApplicationForDelay.is_nature_day.key: kwargs.get('isNatureDay', ''),
GovsApplicationForDelay.already_notify_order_user.key: kwargs.get('alreadyNotifyOrderUser', ''),
GovsApplicationForDelay.request_reason.key: kwargs.get('requestReason', ''),
GovsApplicationForDelay.remarks.key: kwargs.get('remarks', ''),
GovsApplicationForDelay.contact_name.key: kwargs.get('contactName', ''),
GovsApplicationForDelay.contact_time.key: kwargs.get('contactTime', ''),
GovsApplicationForDelay.contact_type.key: kwargs.get('contactType', ''),
GovsApplicationForDelay.contact_type_name.key: kwargs.get('contactTypeName', ''),
GovsApplicationForDelay.reply_script.key: kwargs.get('replyScript', ''),
GovsApplicationForDelay.file_id_str.key: kwargs.get('fileIdStr', ''),
GovsApplicationForDelay.request_delay_time.key: kwargs.get('requestDelayTime', ''),
GovsApplicationForDelay.flow_token.key: kwargs.get('flowToken', '')
}
async def create_delay(self, **kwargs) -> dict:
# 必填参数校验
required_keys = [
'gdId', 'flowToken', 'finallyTimeAfterApprove', 'requestDelay', 'isNatureDay',
'alreadyNotifyOrderUser', 'requestReason'
]
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
govs_task_id = kwargs.get('gdId', '')
self.govs_order = await GovsOrderMaster.async_find_by_id(govs_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.govs_delay = GovsApplicationForDelay().copy_from_dict(params)
self.govs_delay.status = 0
await self.govs_delay.async_save()
# 后台执行提交申请延期请求到省12345
await aio_pool.run_background_task(
govs_create_order_delay.create_delay(self.govs_delay, self.govs_order)
)
return {
'msg': '申请延期成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 申请延期接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.create_delay(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+102
View File
@@ -0,0 +1,102 @@
from typing import Optional
import logging
from apps.api import govs
from apps.app_handler import AppHandler
from paste.web.decorators import route
from paste.core import aio_pool
from paste.core.logging import echo_log
from dock.govs import govs_create_order_return
from models.govs_order_master import GovsOrderMaster
from models.govs_create_return import GovsWorkOrderReturnFormal
@route(f'{govs.ApiPrefix}/work-order-return-formal/create')
class CreateDelayHandler(AppHandler):
"""
申请工单退回接口。
对接省12345的申请退回接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.govs_order: Optional[GovsOrderMaster] = None
self.govs_return: Optional[GovsWorkOrderReturnFormal] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
GovsWorkOrderReturnFormal.master_id.key: kwargs.get('gdId', ''),
GovsWorkOrderReturnFormal.flow_token.key: kwargs.get('flowToken', ''),
GovsWorkOrderReturnFormal.gd_id.key: kwargs.get('gdId', ''),
GovsWorkOrderReturnFormal.return_reason.key: kwargs.get('returnReason', ''),
GovsWorkOrderReturnFormal.return_reason_name.key: kwargs.get('returnReasonName', ''),
GovsWorkOrderReturnFormal.return_auditor_name.key: kwargs.get('returnAuditorName', ''),
GovsWorkOrderReturnFormal.return_auditor_id.key: kwargs.get('returnAuditorId', ''),
GovsWorkOrderReturnFormal.deal_opinion.key: kwargs.get('dealOpinion', ''),
GovsWorkOrderReturnFormal.reason.key: kwargs.get('reason', ''),
GovsWorkOrderReturnFormal.remark.key: kwargs.get('remark', ''),
GovsWorkOrderReturnFormal.file_id_str.key: kwargs.get('fileIdStr', ''),
GovsWorkOrderReturnFormal.process_instance_id.key: kwargs.get('processInstanceId', ''),
GovsWorkOrderReturnFormal.action_name.key: kwargs.get('actionName', ''),
GovsWorkOrderReturnFormal.order_id.key: kwargs.get('orderId', ''),
GovsWorkOrderReturnFormal.task_id.key: kwargs.get('taskId', ''),
GovsWorkOrderReturnFormal.order_no.key: kwargs.get('orderNo', ''),
GovsWorkOrderReturnFormal.case_accord_type_one_name.key: kwargs.get('caseAccordTypeOneName', ''),
GovsWorkOrderReturnFormal.case_accord_type_two_name.key: kwargs.get('caseAccordTypeTwoName', ''),
GovsWorkOrderReturnFormal.case_accord_type_three_name.key: kwargs.get('caseAccordTypeThreeName', '')
}
async def create_return(self, **kwargs) -> dict:
# 必填参数校验
required_keys = [
'gdId', 'flowToken', 'returnReason', 'returnReasonName', 'dealOpinion', 'reason'
]
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
govs_task_id = kwargs.get('gdId', '')
self.govs_order = await GovsOrderMaster.async_find_by_id(govs_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.govs_return = GovsWorkOrderReturnFormal().copy_from_dict(params)
self.govs_return.status = 0
await self.govs_return.async_save()
# 后台执行提交申请延期请求到省12345
await aio_pool.run_background_task(
govs_create_order_return.create_return(self.govs_return, self.govs_order)
)
return {
'msg': '申请退回成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 申请退回接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.create_return(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+102
View File
@@ -0,0 +1,102 @@
from typing import Optional
import logging
from apps.api import govs
from apps.app_handler import AppHandler
from paste.web.decorators import route
from paste.core import aio_pool
from paste.core.logging import echo_log
from dock.govs import govs_create_reply
from models.govs_order_master import GovsOrderMaster
from models.govs_create_reply import GovsReplyFormal
@route(f'{govs.ApiPrefix}/reply-formal/create')
class CreateDelayHandler(AppHandler):
"""
答复办结接口。
对接省12345的答复办结接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.govs_order: Optional[GovsOrderMaster] = None
self.govs_reply: Optional[GovsReplyFormal] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
GovsReplyFormal.master_id.key: kwargs.get('gdId', ''),
GovsReplyFormal.flow_token.key: kwargs.get('flowToken', ''),
GovsReplyFormal.gd_id.key: kwargs.get('gdId', ''),
GovsReplyFormal.is_contact.key: kwargs.get('isContact', ''),
GovsReplyFormal.contact_name.key: kwargs.get('contactName', ''),
GovsReplyFormal.contact_time.key: kwargs.get('contactTime', ''),
GovsReplyFormal.contact_type.key: kwargs.get('contactType', ''),
GovsReplyFormal.advice.key: kwargs.get('advice', ''),
GovsReplyFormal.reason.key: kwargs.get('reason', ''),
GovsReplyFormal.remarks.key: kwargs.get('remarks', ''),
GovsReplyFormal.file_id_str.key: kwargs.get('fileIdStr', ''),
GovsReplyFormal.save_id.key: kwargs.get('saveId', ''),
GovsReplyFormal.process_instance_id.key: kwargs.get('processInstanceId', ''),
GovsReplyFormal.business_key.key: kwargs.get('businessKey', ''),
GovsReplyFormal.order_no.key: kwargs.get('orderNo', ''),
GovsReplyFormal.action_name.key: kwargs.get('actionName', ''),
GovsReplyFormal.case_accord_type_one_name.key: kwargs.get('caseAccordTypeOneName', ''),
GovsReplyFormal.case_accord_type_two_name.key: kwargs.get('caseAccordTypeTwoName', ''),
GovsReplyFormal.case_accord_type_three_name.key: kwargs.get('caseAccordTypeThreeName', ''),
}
async def create_delay(self, **kwargs) -> dict:
# 必填参数校验
required_keys = [
'gdId', 'flowToken', 'isContact', 'contactType', 'advice', 'reason'
]
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
govs_task_id = kwargs.get('gdId', '')
self.govs_order = await GovsOrderMaster.async_find_by_id(govs_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.govs_reply = GovsReplyFormal().copy_from_dict(params)
self.govs_reply.status = 0
await self.govs_reply.async_save()
# 后台执行提交答复办结请求到省12345
await aio_pool.run_background_task(
govs_create_reply.create_reply(self.govs_reply, self.govs_order)
)
return {
'msg': '答复办结成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 答复办结接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.create_delay(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+101
View File
@@ -0,0 +1,101 @@
from typing import Optional
import logging
from apps.api import govs
from apps.app_handler import AppHandler
from paste.web.decorators import route
from paste.core import aio_pool
from paste.core.logging import echo_log
from dock.govs import govs_phase_wise_completion
from models.govs_order_master import GovsOrderMaster
from models.govs_phase_wise_completion import GovsPhaseWiseCompletion
@route(f'{govs.ApiPrefix}/phase-wise-completion/create')
class CreateDelayHandler(AppHandler):
"""
阶段性办结接口。
对接省12345的阶段性办结接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.govs_order: Optional[GovsOrderMaster] = None
self.phase_wise_completion: Optional[GovsPhaseWiseCompletion] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
GovsPhaseWiseCompletion.master_id.key: kwargs.get('gdId', ''),
GovsPhaseWiseCompletion.flow_token.key: kwargs.get('flowToken', ''),
GovsPhaseWiseCompletion.gd_id.key: kwargs.get('gdId', ''),
GovsPhaseWiseCompletion.is_contact.key: kwargs.get('isContact', ''),
GovsPhaseWiseCompletion.contact_name.key: kwargs.get('contactName', ''),
GovsPhaseWiseCompletion.contact_time.key: kwargs.get('contactTime', ''),
GovsPhaseWiseCompletion.contact_type.key: kwargs.get('contactType', ''),
GovsPhaseWiseCompletion.next_feedback_time.key: kwargs.get('nextFeedbackTime', ''),
GovsPhaseWiseCompletion.advice.key: kwargs.get('advice', ''),
GovsPhaseWiseCompletion.reason.key: kwargs.get('reason', ''),
GovsPhaseWiseCompletion.remark.key: kwargs.get('remark', ''),
GovsPhaseWiseCompletion.action_name.key: kwargs.get('actionName', ''),
GovsPhaseWiseCompletion.case_accord_type_one_name.key: kwargs.get('caseAccordTypeOneName', ''),
GovsPhaseWiseCompletion.case_accord_type_two_name.key: kwargs.get('caseAccordTypeTwoName', ''),
GovsPhaseWiseCompletion.case_accord_type_three_name.key: kwargs.get('caseAccordTypeThreeName', ''),
GovsPhaseWiseCompletion.order_id.key: kwargs.get('orderId', ''),
GovsPhaseWiseCompletion.task_id.key: kwargs.get('taskId', '')
}
async def create_delay(self, **kwargs) -> dict:
# 必填参数校验
required_keys = [
'gdId', 'flowToken', 'isContact', 'contactName', 'contactTime', 'contactType', 'nextFeedbackTime', 'advice',
'reason'
]
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
govs_task_id = kwargs.get('gdId', '')
self.govs_order = await GovsOrderMaster.async_find_by_id(govs_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.phase_wise_completion = GovsPhaseWiseCompletion().copy_from_dict(params)
self.phase_wise_completion.status = 0
await self.phase_wise_completion.async_save()
# 后台执行提交阶段性办结请求到省12345
await aio_pool.run_background_task(
govs_phase_wise_completion.create_phase_wise_completion(self.phase_wise_completion, self.govs_order)
)
return {
'msg': '阶段性办结成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 阶段性办结接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.create_delay(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+91
View File
@@ -0,0 +1,91 @@
from typing import Optional
import logging
from apps.api import govs
from apps.app_handler import AppHandler
from paste.web.decorators import route
from paste.core import aio_pool
from paste.core.logging import echo_log
from dock.govs import govs_save_sign
from models.govs_order_master import GovsOrderMaster
from models.govs_save_sign import GovsSaveSign
@route(f'{govs.ApiPrefix}/save-sign-for/create')
class CreateDelayHandler(AppHandler):
"""
申请延期接口。
对接省12345的申请延期接口,请求后本接口先将数据保存本地,然后响应客户端,然后开始后台启动推送。
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.govs_order: Optional[GovsOrderMaster] = None
self.govs_sign: Optional[GovsSaveSign] = None
def _params_for_db(self, **kwargs: dict) -> dict:
"""
提取数据库所需参数。
"""
return {
GovsSaveSign.gd_id.key: kwargs.get('gdId', ''),
GovsSaveSign.flow_token.key: kwargs.get('flowToken', ''),
GovsSaveSign.order_id.key: kwargs.get('orderId', ''),
GovsSaveSign.order_no.key: kwargs.get('orderNo', ''),
GovsSaveSign.master_id.key: kwargs.get('gdId', ''),
GovsSaveSign.order_process_id.key: kwargs.get('orderProcessId', ''),
GovsSaveSign.task_id.key: kwargs.get('taskId', ''),
GovsSaveSign.flag.key: kwargs.get('flag', '')
}
async def create_delay(self, **kwargs) -> dict:
# 必填参数校验
required_keys = [
'gdId', 'flowToken'
]
missing = [
k for k in required_keys
if k not in kwargs or kwargs[k] is None
]
if missing:
raise ValueError(f"缺少必要参数: {missing}")
# 读取待办任务对象
govs_task_id = kwargs.get('gdId', '')
self.govs_order = await GovsOrderMaster.async_find_by_id(govs_task_id)
# 保存请求数据
params = self._params_for_db(**kwargs)
self.govs_sign = GovsSaveSign().copy_from_dict(params)
self.govs_sign.status = 0
await self.govs_sign.async_save()
# 后台执行提交申请延期请求到省12345
await aio_pool.run_background_task(
govs_save_sign.sign_order(self.govs_sign, self.govs_order)
)
return {
'msg': '确认签收成功.'
}
# @auth_token
async def post(self):
"""
处理 POST 请求。
---
tags:
- D3I API
summary: 申请延期接口
"""
try:
echo_log(self.request.body.decode())
_, params = self.get_request_params()
_result = await self.create_delay(**params)
self.response_ok(code=0, data=_result)
except Exception as e:
self.response_error(e, status_code=200, api_status_code=500)
self.log(msg=e, level=logging.ERROR, is_log_exc=True)
+183
View File
@@ -0,0 +1,183 @@
import datetime
import json
import os
from abc import ABC
from typing import Optional, Callable, Awaitable
from paste.rbac.rbac_user import RbacUser
from paste.util.encoder import JsonDumpsEncoder
from paste.web.handler import RequestHandler
from paste.web.param_aware_loader import ParamAwareLoader
class AppHandler(RequestHandler, ABC):
"""
控制器基类。
"""
commands: dict[str, Callable] = {}
"""
API 接口命令字典,其结构为命令名称指向对应的方法。
其结构如下::
{
command_name: method
}
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.user: Optional[RbacUser] = None
"""
当前登录用户对象。
"""
self.start_at = datetime.datetime.now()
"""
实例初始化时间。
"""
self.command = ""
"""
命令。
"""
self.request_params = {}
"""
命令参数。
"""
async def after_auth_token(self, token_payload: dict):
"""
初始化登录用户信息。
"""
from paste.security import token
if token_payload != token.PRIVATE_ISS:
from jwt import InvalidTokenError
raise InvalidTokenError()
async def run_command(self):
"""
根据请求参数运行命令方法,返回命令执行结果。
"""
self.get_request_params()
assert self.command in self.commands, '请提供正确的命令参数.'
# 读取命令方法对象,并执行
_cmd_func = self.commands[self.command]
_result = _cmd_func(self, **self.request_params)
# 处理异步方法执行
if isinstance(_result, Awaitable):
_result = await _result
return _result
async def gen_html(self, template_file: str, **kwargs):
"""
生成 HTML 内容。
:param template_file: 模板文件
:param kwargs: 参数数据字典
:return: 返回生成的 HTML 文件内容
"""
# 将参数字典转换为 namedtuple,名称固定
template_data_obj = self.dict_to_namedtuple('TemplateData', {**kwargs})
# 手动构建完整命名空间,加入自定义参数,传给生成器
namespace = self.get_template_namespace()
namespace.update({'td': template_data_obj})
# 获取模板文件
template_file = f"{self.application.settings.get('template_path')}/{template_file}"
# 用参数感知模板加载器,加载模板文件,并传入 namespace 以便在加载完成后,执行数据准备
loader = ParamAwareLoader(os.path.dirname(template_file), namespace=namespace)
# 从文件中加载模板,同步完成数据准备
template = await loader.load_with_prepare(os.path.basename(template_file))
# 渲染模板,传入需要的数据
output = template.generate(**namespace)
return output
def response_ok(self, **kwargs):
self.log_request_end()
super().response_ok(**kwargs)
def response_error(self, e: Exception, status_code: int = 200, api_status_code: int = None, **kwargs):
self.log_request_end()
if api_status_code is None:
api_status_code = status_code
self.set_status(status_code=status_code)
chunk = {'code': api_status_code, 'status': 'error'}
chunk.update(kwargs)
if len(e.args) > 0 and isinstance(e.args[0], str):
chunk['msg'] = e.args[0]
if len(e.args) > 1:
if isinstance(e.args[1], dict):
chunk.update(e.args[1])
elif isinstance(e.args[1], list):
chunk['errors'] = e.args[1]
self.write(json.dumps(chunk, cls=JsonDumpsEncoder, ensure_ascii=False))
self.set_header('Content-Type', 'application/json')
def get_request_params(self):
"""
读取命令名称及请求参数。注意,参数命名应当避开 cmd 和 params。
该方法自动合并参数,并输出请求开始日志。
支持通过 Form 或 Json 两种方式提交请求并读取相应参数。
如使用 Form 方式,则应当在 form-data 中包含名为 cmd 的输入项,其值为对应的命令,其他输入项为命令参数。
注意:不会自动读取上传的文件数据,可通过::
self.request.files
方法读取上传的文件。
如使用 Json 方式,则应当遵循以下结构::
{
cmd: command_name,
params:
{
key: value
}
}
:return: 命令,命令对应的参数
"""
_arguments = self.request_arguments()
_cmd = _arguments.get('cmd', None)
_params = _arguments.get('params', None)
if _params is None:
_arguments.pop('cmd', None)
_params = _arguments
self.command = _cmd
self.request_params = _params
# 合成规则参数到参数字典,规则参数可在相应规则中修改
self.request_params.update(self.rule_kwargs)
# 取得命令和参数之后,记录请求开始日志
self.log_request_start()
return self.command, self.request_params
def log_request_end(self):
end_at = datetime.datetime.now()
total_delta = (end_at - self.start_at).total_seconds()
_spend = f"耗时:{total_delta:f} 秒."
_user_name = self.user.username if self.user else 'Unknown'
_log = f"O 用户:{_user_name} 完成 {self.request.uri}"
_log = f"{_log} 接口命令 {self.command}{_spend}" if self.command else f"{_log} 请求,{_spend}"
self.log(_log)
def log_request_start(self):
"""
收到请求时记录的日志
"""
_user_name = self.user.username if self.user else 'Unknown'
_log = f"I 用户:{_user_name} 请求 {self.request.uri}"
_log = f"{_log} 接口命令 {self.command}." if self.command else f"{_log}."
self.log(_log)
View File
Binary file not shown.
Binary file not shown.
+54
View File
@@ -0,0 +1,54 @@
import datetime
from sqlalchemy.event import listen
from sqlalchemy.pool import Pool, QueuePool
from paste.db import engine
AsyncPoolMaxCheckOut = 0
"""
异步连接池最大连接数量。
"""
AsyncPoolMaxCheckOutAt = datetime.datetime.now()
"""
异步连接池最大连接数量发生时间。
"""
GlobalPoolMaxCheckOut = 0
"""
普通连接池最大连接数量。
"""
GlobalPoolMaxCheckOutAt = datetime.datetime.now()
"""
普通连接池最大连接数量发生时间。
"""
def on_checkout(dbapi_connection, connection_record, connection_proxy):
"""
当取用连接池中的连接后,立即执行的事件。这里用来记录取用峰值数据。
"""
async_pool: QueuePool = engine.async_connect_engine().pool
global_pool = engine.connect_engine().pool
assert isinstance(global_pool, QueuePool), f"引擎类型错误."
global AsyncPoolMaxCheckOut
global AsyncPoolMaxCheckOutAt
global GlobalPoolMaxCheckOut
global GlobalPoolMaxCheckOutAt
_async_checkout = async_pool.checkedout()
if AsyncPoolMaxCheckOut < _async_checkout:
AsyncPoolMaxCheckOut = _async_checkout
AsyncPoolMaxCheckOutAt = datetime.datetime.now()
_global_checkout = global_pool.checkedout()
if GlobalPoolMaxCheckOut < _global_checkout:
GlobalPoolMaxCheckOut = _global_checkout
GlobalPoolMaxCheckOutAt = datetime.datetime.now()
def bind_listener():
listen(Pool, 'checkout', on_checkout)
+360
View File
@@ -0,0 +1,360 @@
"""
控制台工具。
包含 Web 服务管理、远程数据获取服务管理、数据导入命令、用户管理命令、权限设置命令等。
可直接在控制台终端实现部分数据操作与管理。
"""
import argparse
import datetime
import logging
import os.path
import re
import sys
from typing import Optional, Union
from dateutil.relativedelta import relativedelta
import apps
from dock.dcm import dcm_security
from dock.govs import govs_security
from dock.oa import oa_security
from paste.core import aio_pool
from paste.core.logging import echo_log
from paste.db import gen_models
from paste.security import token
from paste.service import server
from paste.util import udict
ArgParser = argparse.ArgumentParser(
prefix_chars='-+',
description=f"命令行管理工具。",
)
"""
命令行参数解析器。
"""
ArgSubparsers = ArgParser.add_subparsers(
dest='sub_command',
title=f'内部命令',
description=f'请使用以下命令以及对应选项完成具体任务。',
help='使用 python3 cli.py [command] -h 查看使用说明。'
)
"""
内部命令。
"""
ArgNamespace: Optional[argparse.Namespace] = None
"""
命令行参数解析结果。
"""
HadBehavior = False
"""
是否正确执行了行为函数。
"""
def had_behavior(success: Optional[bool] = None):
"""
是否成功执行了行为方法,可取得或设置执行状态。
:param success: 执行状态
:return: 执行状态
"""
global HadBehavior
if success is not None:
HadBehavior = success
return HadBehavior
def current_dir():
print(f"当前目录:{os.path.abspath(os.path.curdir)}")
def version():
print(apps.get_version())
def generate_models():
"""
重建数据模型。
"""
try:
_runner = aio_pool.get_aio_runner()
_runner(gen_models.sqlacodegen())
echo_log(msg='成功重建数据模型')
except Exception as e:
echo_log(msg=e, level=logging.ERROR, is_log_exc=True)
def generate_token(client_id: str = 'Any'):
"""
生成一个长期 Token 默认 50 年。
:params client_id: 任意
"""
try:
_now = datetime.datetime.now(datetime.timezone.utc)
_exp = _now + relativedelta(years=50)
_token = token.encode_token(exp=_exp, client_id=client_id)
echo_log(msg=f"过期时间:{_exp}")
echo_log(msg=_token)
except Exception as e:
echo_log(msg=e, level=logging.ERROR, is_log_exc=True)
def renew_token():
_runner = aio_pool.get_aio_runner()
try:
echo_log(f"开始执行数字城管 Cookies 更新...")
_runner(dcm_security.login())
echo_log(f"完成数字城管 Cookies 更新.")
except Exception as e:
echo_log(msg=e, level=logging.ERROR, is_log_exc=True)
try:
echo_log(f"开始执行 OA Token 更新...")
_runner(oa_security.login())
echo_log(f"完成 OA Token 更新.")
except Exception as e:
echo_log(msg=e, level=logging.ERROR, is_log_exc=True)
try:
echo_log(f"开始执行省12345 Token 更新...")
_runner(govs_security.login())
echo_log(f"完成省12345 Token 更新.")
except Exception as e:
echo_log(msg=e, level=logging.ERROR, is_log_exc=True)
def push_task(task_id: Union[str, int]):
"""
单条推送测试。
:param task_id: 待办任务 ID
"""
async def _push_task(_task_id: Union[str, int]):
from dock.oa_dcm.oa_push_order import push_order
await push_order(task_id=_task_id)
from dock.oa_dcm.oa_push_order_detail import push_order_detail
await push_order_detail(task_id=_task_id)
from dock.oa_dcm.oa_push_extend_info import push_extend_info
await push_extend_info(task_id=_task_id)
from dock.oa_dcm.oa_push_more_info import push_more_info
await push_more_info(task_id=_task_id)
from dock.oa_dcm.oa_upload import upload_with_attachment
await upload_with_attachment(task_id=_task_id)
from dock.oa_dcm.oa_push_attachment import push_attachment
await push_attachment(task_id=_task_id)
from dock.oa_dcm.oa_push_process_info import push_process_info
await push_process_info(task_id=_task_id)
# 推送结束后,签收这些已推送的工单
from dock.oa_dcm.oa_sign_task import sign_task
await sign_task(task_id=_task_id)
_runner = aio_pool.get_aio_runner()
_runner(_push_task(task_id))
def start_service(full_service_name):
"""
在控制台启动服务,当控制台关闭时,服务停止。
:param full_service_name: 完整服务名称:package.module
"""
server.start_service(full_service_name)
def start(full_service_name):
"""
启动常驻内存服务。
:param full_service_name: 完整服务名称:package.module
"""
server.start(full_service_name)
def stop(full_service_name):
"""
停止常驻内存服务。
:param full_service_name: 完整服务名称:package.module
"""
server.stop(full_service_name)
tool_config = {
'cmds': {
'generate_models': {
'behavior': generate_models,
'args': {},
'opts': {
'help': f"重建(覆盖)数据模型,注意:第一、用户模型与 RbacUserModel 模型冲突,须删除,注意其外键引用问题。"
},
},
'generate_token': {
'behavior': generate_token,
'args': {
'client_id': {'action': 'store', 'nargs': '?', 'help': '如: backend.0ad8d20fbd2804c038f19c9f522010d3'},
},
'opts': {
'help': f"创建长期 Token。"
},
},
'renew_token': {
'behavior': renew_token,
'args': {},
'opts': {
'help': f"重建Token。"
},
},
'push_task': {
'behavior': push_task,
'args': {
'task_id': {'action': 'store', 'help': '如: 2054531648254513152'},
},
'opts': {
'help': f"推送待办工单。"
},
},
'start_service': {
'behavior': start_service,
'args': {
'full_service_name': {'action': 'store', 'help': '如 package.module 格式的服务名称'},
},
'opts': {
'help': f"在控制台启动服务,注意:当控制台关闭时,服务随即停止"
},
},
'start': {
'behavior': start,
'args': {
'full_service_name': {'action': 'store', 'help': '如 package.module 格式的服务名称'},
},
'opts': {
'help': f"常驻内存方式启动服务,当控制台关闭时,服务将继续运行,直到调用 stop 命令"
},
},
'stop': {
'behavior': stop,
'args': {
'full_service_name': {'action': 'store', 'help': '如 package.module 格式的服务名称'},
},
'opts': {
'help': f"停止服务"
},
},
},
'args': {
'-d': {
'opts': {'action': 'store_true', 'help': '显示当前目录'},
'behavior': {'callback': current_dir, 'args': []}
},
'-v': {
'opts': {'action': 'store_true', 'help': '系统版本'},
'behavior': {'callback': version, 'args': []}
},
}
}
"""
命令行以及内部命令详细配置。
"""
def init_argument():
"""
解析命令行参数,填充命令参数名字空间。
"""
# 初始化内部命令
for _cmd_name, _command in tool_config.get('cmds', {}).items():
_parser = ArgSubparsers.add_parser(_cmd_name, **_command.get('opts', {}))
for _arg_name, _argument in _command.get('args', {}).items():
assert isinstance(_argument, dict)
_parser.add_argument(_arg_name, **_argument)
# 初始化选项
for _arg_name, _argument in tool_config.get('args', {}).items():
ArgParser.add_argument(_arg_name, **_argument.get('opts', {}))
global ArgNamespace
ArgNamespace = ArgParser.parse_args(sys.argv[1:])
def arg(arg_name: str):
"""
从命令参数名字空间中获取值。
:param arg_name: 选项名称
:return: 选项值
"""
global ArgNamespace
return getattr(ArgNamespace, arg_name, None)
def exec_behavior():
"""
执行行为方法。
"""
sub_command_name = arg('sub_command')
sub_command = udict.get_by_path(tool_config, f'cmds.{sub_command_name}', None)
if sub_command_name is not None and sub_command is not None:
# 执行命令行为
# 回调函数
_callback = udict.get_by_path(sub_command, 'behavior', None)
if _callback is not None and callable(_callback):
# 形参列表
_arg_names = udict.get_by_path(sub_command, 'args', {}).keys()
# 取得实参
_kwargs = {}
for _name in _arg_names:
_k = _name.lstrip('-')
_kwargs[_k] = arg(_k)
# 验证并执行回调函数
if len(_kwargs) == len(_arg_names):
had_behavior(True)
_callback(**_kwargs)
else:
# 执行选项行为
for _arg_name, _argument in tool_config.get('args', {}).items():
_name = re.sub(r'[-+/]', '', _arg_name)
if arg(_name) is None or not arg(_name):
# 未包含选项,跳过
continue
_behavior = _argument.get('behavior', None)
if _behavior is None:
# 未配置行为,跳出
continue
# 回调函数
_callback = _behavior.get('callback', None)
if _callback is None or not hasattr(_callback, '__call__'):
# 行为是否是一个可用的方法或函数
continue
# 形参列表
_arg_names = _behavior.get('args', [])
# 取得实参
_kwargs = [arg(_name) for _name in _arg_names]
# 验证并执行回调函数
if callable(_callback) and len(_kwargs) == len(_arg_names):
had_behavior(True)
_callback(*_kwargs)
if not had_behavior():
ArgParser.print_help()
if __name__ == '__main__':
init_argument()
exec_behavior()
+215
View File
@@ -0,0 +1,215 @@
"""
公共对接模块。
"""
import asyncio
import random
import re
from http.cookies import SimpleCookie
from typing import Optional, Dict, Any
from tornado.httpclient import HTTPRequest, HTTPResponse
from paste.web import requests
USER_AGENTS = [
# Chrome on Windows 10
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
# Firefox on Windows 10
'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0',
# Edge on Windows 11 (Chromium-based)
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.2210.91',
# Safari on macOS
'Mozilla/5.0 (Macintosh; Intel Mac OS X 14_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15',
# Chrome on macOS
'Mozilla/5.0 (Macintosh; Intel Mac OS X 14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
# Firefox on macOS
'Mozilla/5.0 (Macintosh; Intel Mac OS X 14.2; rv:121.0) Gecko/20100101 Firefox/121.0',
# Chrome on Linux
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
# Opera on Windows (Chromium-based)
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 OPR/106.0.0.0',
]
"""
常用 PC 端浏览器 User-Agent 列表。
"""
DEFAULT_TIMEOUT = 120.0
"""
默认超时时长。
"""
CONCURRENCY_COUNT = 5
"""
最大并发次数。
"""
MAX_RETRY_COUNT = 5
"""
最大重试次数。
"""
def get_random_user_agent() -> tuple[str, str, str]:
"""
从 user_agents 列表中随机返回一个 User-Agent 字符串及其浏览器版本和操作系统名称。
Returns:
tuple: (user_agent: str, browser_version: str, os_name: str)
"""
ua: str = random.choice(USER_AGENTS)
# 提取浏览器版本
browser_version: str = "Unknown"
if "Chrome/" in ua:
match = re.search(r"Chrome/(\d+\.\d+\.\d+\.\d+)", ua)
if match:
browser_version = match.group(1)
elif "Firefox/" in ua:
match = re.search(r"Firefox/(\d+\.\d+)", ua)
if match:
browser_version = match.group(1)
elif "Version/" in ua and "Safari/" in ua: # Safari
match = re.search(r"Version/(\d+\.\d+)", ua)
if match:
browser_version = match.group(1)
elif "Edg/" in ua:
match = re.search(r"Edg/(\d+\.\d+\.\d+\.\d+)", ua)
if match:
browser_version = match.group(1)
elif "OPR/" in ua:
match = re.search(r"OPR/(\d+\.\d+\.\d+\.\d+)", ua)
if match:
browser_version = match.group(1)
# 提取操作系统名称
os_name: str = "Unknown"
if "Mac" in ua:
os_name = "Mac"
elif "Windows" in ua:
os_name = "Windows"
elif "Linux" in ua:
os_name = "Linux"
return ua, browser_version, os_name
def get_cookies(response: HTTPResponse) -> Dict[str, str]:
"""
从响应对象读取 Cookies。
:param response: 请求响应对象
:return: 提取到的 Cookies
"""
cookies = SimpleCookie()
for set_cookie in response.headers.get_list('Set-Cookie'):
cookies.load(set_cookie)
return {k: v.value for k, v in cookies.items()}
def get_cookie_value(cookies_string, cookie_name):
"""
从 cookies 字符串中按名称提取对应的值
参数:
cookies_string: str, 格式为 "key1=value1; key2=value2; ..."
cookie_name: str, 要查找的 cookie 名称
返回:
str 或 None: 如果找到返回对应的值,否则返回 None
"""
# 按分号分割 cookie 字符串
cookies_list = cookies_string.split(';')
for cookie in cookies_list:
# 去除前后空格,然后按等号分割键值对
parts = cookie.strip().split('=', 1)
if len(parts) == 2:
key, value = parts
if key == cookie_name:
return value
# 如果没有找到,返回 None
return None
def new_http_request(
url: str,
body: Optional[Dict[str, Any]] = None,
method: str = 'POST',
timeout: Optional[float] = None,
follow_redirects: bool = True,
use_form: bool = False,
extra_headers: Optional[Dict[str, str]] = None,
**kwargs
) -> HTTPRequest:
"""
新建 HTTPRequest 对象。
支持 GET 和 POST 方法:
- GET: 参数通过 URL 查询字符串传递
- POST: 参数通过 JSON body 或 form 表单传递(由 use_form 控制)
:param url: 请求的完整 URL
:param body: 请求体(字典),GET 时为查询参数,POST 时为 JSON 或 form 数据
:param method: HTTP 方法,仅支持 'GET''POST'
:param timeout: 请求超时时间(秒)
:param follow_redirects: 是否跟随重定向
:param use_form: 如果为 TruePOST 时使用 application/x-www-form-urlencoded 格式;否则使用 JSON
:param extra_headers: 可选的额外请求头,用于传入 Cookie、Authorization 等
:param kwargs: 其他参数,符合 tornado.httpclient.HTTPRequest 参数要求
:return: tornado.httpclient.HTTPRequest 对象
:raises ValueError: 当 method 不合法时抛出
"""
return requests.build_http_request(
url, body, method, timeout, follow_redirects, use_form, extra_headers,
**kwargs
)
async def scrape_cookies(url: str, timeout: Optional[float] = 10,
extra_headers: Optional[Dict[str, str]] = None,
**kwargs) -> Dict[str, str]:
"""
发送 GET 请求到 url 以获取服务端下发的 Cookies(如 JSESSIONID)。
不关心响应体,只提取响应头中的 Set-Cookie。
返回解析后的 Cookie 字典 { 'name': 'value', ... }
:param url: 获取 Cookies 的路径
:param timeout: 超时时间
:param extra_headers: 扩展头
:return: Cookies 读取到的 Cookies
"""
cookies: Optional[Dict[str, str]] = None
request = new_http_request(
url=url,
method='GET',
timeout=timeout,
follow_redirects=True,
extra_headers=extra_headers,
**kwargs
)
def after_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
nonlocal cookies
cookies = get_cookies(response)
request_queue = asyncio.Queue()
await request_queue.put(request)
await requests.async_concurrency(
request_queue, con_count=1, retry=MAX_RETRY_COUNT,
after_request=after_request
)
if cookies is None:
# 无正确响应,抛出异常
raise Exception(f"未能读取到 Cookies.")
return cookies
Binary file not shown.
+3
View File
@@ -0,0 +1,3 @@
"""
数字城管对接模块。
"""
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+53
View File
@@ -0,0 +1,53 @@
"""
数字城管对接 API 基础功能。
"""
import dock
ApiUrl = "http://221.224.13.41:28086/eUrbanMIS"
"""
对接 API 根目录。
"""
GisUrl = "http://221.224.13.41:28089/eUrbanGIS"
"""
对接 GIS 地址。暂无实际用处。
"""
async def new_api_request(api_url: str, request_body: dict, method: str = 'POST',
timeout: float = dock.DEFAULT_TIMEOUT, use_form: bool = True, headers: dict = None):
"""
构造一个 API 请求对象
:param api_url: API 地址,以斜杠开头的 URI 地址,非完整 URL
:param request_body: 请求体,即所有请求参数
:param method: 请求提交方式
:param timeout: 超时时长
:param use_form: 是否使用表单(Form)方式提交
:param headers: 头数据,最高优先级
:return: HTTPRequest 对象
"""
# Cookie
from dock.dcm import dcm_security
cookie_header = await dcm_security.get_cookies()
# 构建扩展头
user_agent, browser_ver, os_name = dock.get_random_user_agent()
extra_headers = {
'Cookie': cookie_header,
'User-Agent': user_agent,
}
if headers is not None:
extra_headers = {**extra_headers, **headers}
# 构造请求对象
request = dock.new_http_request(
url=f"{ApiUrl}{api_url}",
body=request_body,
method=method,
timeout=timeout,
use_form=use_form,
extra_headers=extra_headers,
)
return request
+216
View File
@@ -0,0 +1,216 @@
import asyncio
import datetime
import io
import logging
from tornado.httpclient import HTTPResponse, HTTPRequest
import apps
import dock
from dock.dcm import dcm_api, dcm_push_upload
from dock.oa import oa_api_request, oa_result_notify, PushException
from dock.oa_dcm import oa_update_post_delay
from models.dcm_apply_delay import DcmApplyPostpone
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.util import ufile
from paste.web import requests
async def get_apply_postpone_request(dcm_apply_postpone: DcmApplyPostpone, dcm_task: DcmTask, media_ids: str,
media_types: str, add_num: int):
"""
创建申请延期请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:param dcm_apply_postpone: 申请延期对象
:param dcm_task: 工单对象
:param media_ids: 上传的附件 MediaID 字符串,多个用英文逗号分隔
:param media_types: 上传的附件的MediaType字符串,多个用英文逗号分隔
:param add_num: 上传的附件数
:return: HTTPRequest 对象
"""
api_url = '/home/workflow/applypostpone'
postpone_desc = f'延长 {dcm_apply_postpone.delay_multiple}倍小类时限({dcm_apply_postpone.delay_multiple * 24}小时)'
body_dict = {
'applyActID': int(dcm_apply_postpone.act_id),
'replyPartID': int(dcm_apply_postpone.reply_part_id),
'ardLevel': dcm_apply_postpone.ard_level,
'ardTypeID': int(dcm_apply_postpone.ard_type_id),
'applyMemo': f'{dcm_apply_postpone.apply_memo}\n{postpone_desc}',
'timeNum': dcm_apply_postpone.delay_multiple * 24,
'timeUnit': dcm_apply_postpone.time_unit,
'postponeDate': dcm_apply_postpone.postpone_date,
'addNum': add_num,
'mediaUsage': '申请延期',
'mediaIDs': media_ids,
'relationTypeID': 10,
'relationID': dcm_task.rec_id,
'relationMainID': 21,
'relationSubID': -1,
'tempUsage': 'apply_postpone_tmp',
'mediaTypes': media_types
}
apply_postpone_request = await dcm_api.new_api_request(api_url, body_dict)
return apply_postpone_request
async def on_attachment_download_error(request: HTTPRequest, exc: Exception, retry_queue: asyncio.Queue = None):
"""
下载附件时的出错处理。
:param request: 请求对象
:param exc: 异常对象
:param retry_queue:
:return:
"""
retry = getattr(request, 'retry', 0)
max_retry = getattr(request, 'max_retry', 0)
if retry < max_retry - 1:
# 非最后一次尝试,不处理
return
message = f"下载申请延期附件时发生错误,下载地址:{request.url}"
echo_log(message)
echo_log(exc, logging.ERROR, is_log_exc=True)
# 保存异常
dcm_apply_postpone: DcmApplyPostpone = getattr(request, 'dcm_apply_postpone')
exc_list = getattr(dcm_apply_postpone, 'exc_list')
exc_list.append(PushException(message, dcm_apply_postpone.flow_token, 3))
async def done_attachment_download(response_list: list[HTTPResponse]):
"""
所有附件下载完成执行的处理程序。
:param response_list: 附件下载响应列表
:return: 返回附件字典
"""
# 遍历取出附件数据
date_str = datetime.datetime.now().strftime("%Y%m%d")
files_body: dict[str, io.IOBase] = {}
for i, response in enumerate(response_list):
_type = ufile.inspect_type(response.body)
if isinstance(_type, tuple):
_fn = f"CT{date_str}{i + 1:04d}.{_type[0]}"
elif isinstance(_type, str) and _type:
_fn = f"CT{date_str}{i + 1:04d}.{_type}"
else:
_fn = f"CT{date_str}{i + 1:04d}"
file_obj = io.BytesIO(response.body)
files_body[_fn] = file_obj
return files_body
async def after_push_apply_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('申请延期请求成功.')
async def push_apply_postpone(dcm_apply_postpone: DcmApplyPostpone, dcm_task: DcmTask):
"""
推送申请延期请求。
:param dcm_apply_postpone: 保存在数据库的申请延期对象
:param dcm_task: 待办工单
"""
try:
files_body = {}
if dcm_apply_postpone.attachments:
echo_log(f"正在准备下载队列...")
setattr(dcm_apply_postpone, 'exc_list', [])
# 创建并填充下载请求队列
attachment_download_queue = asyncio.Queue()
attachment_id_list = dcm_apply_postpone.attachments.split(',')
for attachment_id in attachment_id_list:
download_request = await oa_api_request.get_download_request(attachment_id)
setattr(download_request, 'dcm_apply_postpone', dcm_apply_postpone)
await attachment_download_queue.put(download_request)
# 启动下载,得到所有的附件
files_body: dict[str, io.IOBase] = await requests.async_concurrency(
attachment_download_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_done=done_attachment_download, on_error=on_attachment_download_error
)
# 读取异常有则报错
exc_list: list[PushException] = getattr(dcm_apply_postpone, 'exc_list', [])
if exc_list:
message = ";".join([f"{exc}" for exc in exc_list])
raise PushException(message, dcm_apply_postpone.flow_token, 3)
echo_log(f"附件下载完成,正在准备提交数字城管...")
# 仅生产环境真实提交,其他环境不实际提交
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
# 上传附件,同步得到 MediaIds
media_ids, media_types, add_num = await dcm_push_upload.push_upload(
dcm_task, dcm_push_upload.ApplyPostponeTmp, files_body, -1, 21
)
# 创建并填充数字城管请求队列
apply_postpone_request = await get_apply_postpone_request(
dcm_apply_postpone, dcm_task, media_ids, media_types, add_num
)
setattr(apply_postpone_request, 'dcm_apply_postpone', dcm_apply_postpone)
apply_postpone_push_queue = asyncio.Queue()
await apply_postpone_push_queue.put(apply_postpone_request)
# 提交数字城管-申请延期请求
apply_postpone_push_response_list = await requests.async_concurrency(
apply_postpone_push_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_push_apply_request
)
# 检查是否推送成功,失败直接报错
if len(apply_postpone_push_response_list) != 1:
raise PushException("申请延期请求发生错误.", dcm_apply_postpone.flow_token, 3)
# TODO: 这里要做实际的推送成功检查
# 保存成功状态
dcm_apply_postpone.status = 1
await dcm_apply_postpone.async_save()
# 申请延期请求提交后,通知申请延期成功
await oa_result_notify.push_result_notify(
dcm_apply_postpone.flow_token,
'申请延期成功',
1
)
# 更新延期时间
await oa_update_post_delay.update_process_delay(dcm_apply_postpone.dcm_task_id)
except PushException as e:
# 任何异常都意味着失败,通知 OA
echo_log(f'申请延期发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
# 保存失败状态
dcm_apply_postpone.status = 0
await dcm_apply_postpone.async_save()
# 申请延期发生异常,通知申请延期失败
await oa_result_notify.push_result_notify(
e.flow_token, f"{e}", e.return_code
)
except Exception as e:
# 其他异常都意味着失败,通知 OA
echo_log(f'申请延期发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
if __name__ == "__main__":
from paste.core import aio_pool
async def test():
dcm_task = await DcmTask(id=2054174091237265408).async_find_first()
task_apply_postpone = await DcmApplyPostpone(id=2055639783446810624).async_find_first()
await push_apply_postpone(task_apply_postpone, dcm_task)
_runner = aio_pool.get_aio_runner()
_runner(test())
+210
View File
@@ -0,0 +1,210 @@
import asyncio
import datetime
import io
import logging
from tornado.httpclient import HTTPResponse, HTTPRequest
import apps
import dock
from dock.dcm import dcm_api, dcm_push_upload
from dock.oa import oa_api_request, oa_result_notify, PushException
from models.dcm_apply_delay import DcmApplyPostpone
from models.dcm_apply_rollback import DcmApplyRollback
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.util import ufile
from paste.web import requests
async def get_apply_rollback_request(dcm_apply_rollback: DcmApplyRollback, dcm_task: DcmTask, media_ids: str,
media_types: str, add_num: int):
"""
创建申请回退请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:param dcm_apply_rollback: 请求申请回退对象
:param dcm_task: 工单对象
:param media_ids: 上传的附件 MediaID 字符串,多个用英文逗号分隔
:param media_types: 上传的附件的MediaType字符串,多个用英文逗号分隔
:param add_num: 上传的附件数
:return: HTTPRequest 对象
"""
api_url = '/home/workflow/applyrollback'
body_dict = {
'applyActID': dcm_apply_rollback.act_id,
'replyPartID': dcm_apply_rollback.reply_part_id,
'ardLevel': dcm_apply_rollback.ard_level,
'ardTypeID': dcm_apply_rollback.ard_type_id,
'applyMemo': dcm_apply_rollback.opinion,
'transInfo': dcm_apply_rollback.trans_info,
'addNum': add_num,
'mediaUsage': '申请回退',
'mediaIDs': media_ids,
'relationTypeID': 10,
'relationID': dcm_task.rec_id,
'relationMainID': 21,
'relationSubID': -1,
'tempUsage': 'apply_rollback_tmp',
'mediaTypes': media_types
}
apply_rollback_request = await dcm_api.new_api_request(api_url, body_dict)
return apply_rollback_request
async def on_attachment_download_error(request: HTTPRequest, exc: Exception, retry_queue: asyncio.Queue = None):
"""
下载附件时的出错处理。
:param request: 请求对象
:param exc: 异常对象
:param retry_queue:
:return:
"""
retry = getattr(request, 'retry', 0)
max_retry = getattr(request, 'max_retry', 0)
if retry < max_retry - 1:
# 非最后一次尝试,不处理
return
message = f"下载申请回退附件时发生错误,下载地址:{request.url}"
echo_log(message)
echo_log(exc, logging.ERROR, is_log_exc=True)
# 保存异常
dcm_apply_rollback: DcmApplyPostpone = getattr(request, 'dcm_apply_rollback')
exc_list = getattr(dcm_apply_rollback, 'exc_list')
exc_list.append(PushException(message, dcm_apply_rollback.flow_token, 3))
async def done_attachment_download(response_list: list[HTTPResponse]):
"""
所有附件下载完成执行的处理程序。
:param response_list: 附件下载响应列表
:return: 返回附件字典
"""
# 遍历取出附件数据
date_str = datetime.datetime.now().strftime("%Y%m%d")
files_body: dict[str, io.IOBase] = {}
for i, response in enumerate(response_list):
_type = ufile.inspect_type(response.body)
if isinstance(_type, tuple):
_fn = f"CT{date_str}{i + 1:04d}.{_type[0]}"
elif isinstance(_type, str) and _type:
_fn = f"CT{date_str}{i + 1:04d}.{_type}"
else:
_fn = f"CT{date_str}{i + 1:04d}"
file_obj = io.BytesIO(response.body)
files_body[_fn] = file_obj
return files_body
async def after_push_apply_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('申请回退请求成功.')
async def push_apply_rollback(dcm_apply_rollback: DcmApplyRollback, dcm_task: DcmTask):
"""
推送申请回退请求。
:param dcm_apply_rollback: 保存在数据库的申请回退对象
:param dcm_task: 待办工单
"""
try:
files_body = {}
if dcm_apply_rollback.attachments:
echo_log(f"正在准备下载队列...")
setattr(dcm_apply_rollback, 'exc_list', [])
# 创建并填充下载请求队列
attachment_download_queue = asyncio.Queue()
attachment_id_list = dcm_apply_rollback.attachments.split(',')
for attachment_id in attachment_id_list:
download_request = await oa_api_request.get_download_request(attachment_id)
setattr(download_request, 'dcm_apply_rollback', dcm_apply_rollback)
await attachment_download_queue.put(download_request)
# 启动下载,得到所有的附件
files_body: dict[str, io.IOBase] = await requests.async_concurrency(
attachment_download_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_done=done_attachment_download, on_error=on_attachment_download_error
)
# 读取异常有则报错
exc_list: list[PushException] = getattr(dcm_apply_rollback, 'exc_list', [])
if exc_list:
message = ";".join([f"{exc}" for exc in exc_list])
raise PushException(message, dcm_apply_rollback.flow_token, 3)
echo_log(f"附件下载完成,正在准备提交数字城管...")
# 仅生产环境真实提交,其他环境不实际提交
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
# 上传附件,同步得到 MediaIds
media_ids, media_types, add_num = await dcm_push_upload.push_upload(
dcm_task, dcm_push_upload.ApplyRollbackTmp, files_body, -1, 21
)
# 创建并填充数字城管请求队列
apply_rollback_request = await get_apply_rollback_request(
dcm_apply_rollback, dcm_task, media_ids, media_types, add_num
)
setattr(apply_rollback_request, 'dcm_apply_rollback', dcm_apply_rollback)
apply_postpone_push_queue = asyncio.Queue()
await apply_postpone_push_queue.put(apply_rollback_request)
# 提交数字城管-申请回退请求
apply_postpone_push_response_list = await requests.async_concurrency(
apply_postpone_push_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_push_apply_request
)
# 检查是否推送成功,失败直接报错
if len(apply_postpone_push_response_list) != 1:
raise PushException("申请回退请求发生错误.", dcm_apply_rollback.flow_token, 3)
# TODO: 这里要做实际的推送成功检查
# 保存成功状态
dcm_apply_rollback.status = 1
await dcm_apply_rollback.async_save()
# 申请回退请求提交后,通知申请回退成功
await oa_result_notify.push_result_notify(
dcm_apply_rollback.flow_token,
'申请回退成功',
1
)
except PushException as e:
# 任何异常都意味着失败,通知 OA
echo_log(f'申请回退发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
# 保存失败状态
dcm_apply_rollback.status = 0
await dcm_apply_rollback.async_save()
# 申请回退发生异常,通知申请回退失败
await oa_result_notify.push_result_notify(
e.flow_token, f"{e}", e.return_code
)
except Exception as e:
# 其他异常都意味着失败,通知 OA
echo_log(f'申请回退发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
if __name__ == "__main__":
from paste.core import aio_pool
async def test():
dcm_task = await DcmTask(id=2054174091300179971).async_find_first()
task_rollback = await DcmApplyRollback(id=2055564426366554112).async_find_first()
await push_apply_rollback(task_rollback, dcm_task)
_runner = aio_pool.get_aio_runner()
_runner(test())
+74
View File
@@ -0,0 +1,74 @@
import asyncio
import json
import apps
import dock
from dock.dcm import dcm_api
from tornado.httpclient import HTTPResponse, HTTPRequest
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from models.dcm_dispose import DcmDispose
from paste.web import requests
async def get_conv_dispose_request(dcm_dispose: DcmDispose, dcm_task: DcmTask):
"""
创建便民批转请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:param dcm_dispose: 保存在数据库的批转对象
:param dcm_task: 待办工单对象
"""
api_url = '/home/form/formpreview/save'
form_param = {
'recID': dcm_task.rec_id,
'actID': dcm_dispose.act_id
}
component_list = [
{"componentID": 9, "value": dcm_dispose.undertake_user_name}, # 承办人员
{"componentID": 12, "value": dcm_dispose.undertake_phone} # 联系电话
]
body = {
"formID": 377,
"isInsert": "0",
"briefParam": True,
"formParam": json.dumps(form_param, ensure_ascii=False),
"componentList": json.dumps(component_list, ensure_ascii=False)
}
conv_request = await dcm_api.new_api_request(api_url, body)
return conv_request
async def after_conv_dispose_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('便民批转请求成功.')
# TODO: 这里要做实际的推送成功检查
async def push_conv_dispose_request(dcm_dispose: DcmDispose, dcm_task: DcmTask):
"""
推送批转请求。
:param dcm_dispose: 保存在数据库的批转对象
:param dcm_task: 待办工单对象
"""
echo_log(f"正在准备提交便民批转请求...")
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
request = await get_conv_dispose_request(dcm_dispose, dcm_task)
queue = asyncio.Queue()
await queue.put(request)
# 需要让调用方获取extendInfo参数
return await requests.async_concurrency(
queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_conv_dispose_request
)
+75
View File
@@ -0,0 +1,75 @@
import asyncio
import json
import apps
import dock
from dock.dcm import dcm_api
from tornado.httpclient import HTTPResponse, HTTPRequest
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from models.dcm_rollback import DcmRollback
from paste.web import requests
async def get_conv_rollback_request(rollback: DcmRollback, dcm_task: DcmTask):
"""
创建便民回退请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:param rollback: 保存在数据库的回退对象
:param dcm_task: 待办工单对象
"""
api_url = '/home/form/formpreview/save'
form_param = {
'recID': dcm_task.rec_id,
'actID': rollback.act_id
}
component_list = [
{"componentID": 8, "value": rollback.not_assigned_reason}, # 不交办原因
{"componentID": 4, "value": rollback.not_assigned}, # 是否不交办
{"componentID": 9, "value": rollback.undertake_user_name}, # 承办人员
{"componentID": 12, "value": rollback.undertake_phone}, # 联系电话
]
body = {
"formID": 352,
"isInsert": "0",
"briefParam": True,
"formParam": json.dumps(form_param, ensure_ascii=False),
"componentList": json.dumps(component_list, ensure_ascii=False),
}
conv_request = await dcm_api.new_api_request(api_url, body)
return conv_request
async def after_conv_rollback_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('便民回退请求成功')
# TODO: 这里要做实际的推送成功检查
async def push_conv_rollback_request(dcm_rollback: DcmRollback, dcm_task: DcmTask):
"""
推送回退请求。
:param dcm_rollback: 保存在数据库的回退对象
:param dcm_task: 待办工单对象
"""
echo_log(f"正在准备提交便民回退请求...")
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
request = await get_conv_rollback_request(dcm_rollback, dcm_task)
queue = asyncio.Queue()
await queue.put(request)
# 需要让调用方获取extendInfo参数
return await requests.async_concurrency(
queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_conv_rollback_request
)
+233
View File
@@ -0,0 +1,233 @@
import asyncio
import datetime
import io
import logging
import json
from tornado.httpclient import HTTPResponse, HTTPRequest
import apps
import dock
from dock.dcm import dcm_api, dcm_push_conv_dispose, dcm_send_sms, dcm_push_upload
from dock.oa import oa_api_request, PushException, oa_result_notify
from models.dcm_dispose import DcmDispose
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.util import ufile, udict
from paste.web import requests
async def get_dispose_request(dcm_dispose: DcmDispose, dcm_task: DcmTask, media_ids: str, media_types: str,
add_num: int, extra_opinion: str):
"""
创建批转请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:param dcm_dispose: 批转对象
:param dcm_task: 工单对象
:param media_ids: 上传的附件 MediaID 字符串,多个用英文逗号分隔
:param media_types: 上传的附件的MediaType字符串,多个用英文逗号分隔
:param add_num: 上传的附件数
:param extra_opinion: 便民表单的extendInfo文本
:return: HTTPRequest 对象
"""
api_url = '/home/workflow/transit'
body_dict = {
"actID": int(dcm_dispose.act_id),
"taskListID": int(dcm_dispose.task_list_id),
"opinion": dcm_dispose.opinion if not extra_opinion else dcm_dispose.opinion + ' ' + extra_opinion,
"transInfo": dcm_dispose.trans_info,
"addNum": add_num,
'mediaIDs': media_ids,
'mediaUsage': '处置',
'relationTypeID': 10,
'relationID': dcm_task.rec_id,
'relationMainID': 45,
'relationSubID': dcm_dispose.act_id,
'tempUsage': 'transit_tmp',
'mediaTypes': media_types
}
dispose_request = await dcm_api.new_api_request(api_url, body_dict)
return dispose_request
async def on_attachment_download_error(request: HTTPRequest, exc: Exception, retry_queue: asyncio.Queue = None):
"""
下载附件时的出错处理。
:param request: 请求对象
:param exc: 异常对象
:param retry_queue:
:return:
"""
retry = getattr(request, 'retry', 0)
max_retry = getattr(request, 'max_retry', 0)
if retry < max_retry - 1:
# 非最后一次尝试,不处理
return
message = f"下载批转附件时发生错误,下载地址:{request.url}"
echo_log(message)
echo_log(exc, logging.ERROR, is_log_exc=True)
# 保存异常
dcm_dispose: DcmDispose = getattr(request, 'dcm_dispose')
exc_list = getattr(dcm_dispose, 'exc_list')
exc_list.append(PushException(message, dcm_dispose.flow_token, 3))
async def done_attachment_download(response_list: list[HTTPResponse]):
"""
所有附件下载完成执行的处理程序。
:param response_list: 附件下载响应列表
:return: 返回附件字典
"""
# 遍历取出附件数据
date_str = datetime.datetime.now().strftime("%Y%m%d")
files_body: dict[str, io.IOBase] = {}
for i, response in enumerate(response_list):
_type = ufile.inspect_type(response.body)
if isinstance(_type, tuple):
_fn = f"CT{date_str}{i + 1:04d}.{_type[0]}"
elif isinstance(_type, str) and _type:
_fn = f"CT{date_str}{i + 1:04d}.{_type}"
else:
_fn = f"CT{date_str}{i + 1:04d}"
file_obj = io.BytesIO(response.body)
files_body[_fn] = file_obj
return files_body
async def after_push_dispose_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('批转请求成功.')
async def push_dispose(dcm_dispose: DcmDispose, dcm_task: DcmTask):
"""
推送批转请求。
:param dcm_dispose: 保存在数据库的批转对象
:param dcm_task: 待办工单
"""
try:
# 如果来源是12345热线,提交便民批转
extra_opinion = ''
if dcm_task.event_src_name == '12345热线':
conv_response_list = await dcm_push_conv_dispose.push_conv_dispose_request(dcm_dispose, dcm_task)
if conv_response_list:
conv_response = conv_response_list[0].body.decode()
conv_response = json.loads(conv_response)
extra_opinion = udict.get_by_path(conv_response, 'resultInfo.data.extendInfo', '')
echo_log(f"正在准备下载队列...")
setattr(dcm_dispose, 'exc_list', [])
files_body = {}
if dcm_dispose.attachments:
# 创建并填充下载请求队列
attachment_download_queue = asyncio.Queue()
attachment_id_list = dcm_dispose.attachments.split(',')
for attachment_id in attachment_id_list:
download_request = await oa_api_request.get_download_request(attachment_id)
setattr(download_request, 'dcm_dispose', dcm_dispose)
await attachment_download_queue.put(download_request)
# 启动下载,得到所有的附件
files_body: dict[str, io.IOBase] = await requests.async_concurrency(
attachment_download_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_done=done_attachment_download, on_error=on_attachment_download_error
)
# 读取异常有则报错
exc_list: list[PushException] = getattr(dcm_dispose, 'exc_list', [])
if exc_list:
message = ";".join([f"{exc}" for exc in exc_list])
raise PushException(message, dcm_dispose.flow_token, 3)
echo_log(f"附件下载完成,正在准备提交数字城管...")
# 仅生产环境真实提交,其他环境不实际提交
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
# 上传附件,同步得到 MediaIds
media_ids, media_types, add_num = await dcm_push_upload.push_upload(
dcm_task, dcm_push_upload.TransitTmp, files_body
)
# 创建并填充数字城管请求队列
dispose_request = await get_dispose_request(
dcm_dispose, dcm_task, media_ids, media_types, add_num, extra_opinion
)
setattr(dispose_request, 'dcm_dispose', dcm_dispose)
dispose_push_queue = asyncio.Queue()
await dispose_push_queue.put(dispose_request)
# 提交数字城管-批转请求
dispose_push_response_list = await requests.async_concurrency(
dispose_push_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_push_dispose_request
)
# 检查是否推送成功,失败直接报错
if len(dispose_push_response_list) != 1:
raise PushException("批转请求发生错误.", dcm_dispose.flow_token, 3)
# 检查响应的内容,验证是否推送成功
response_data = dispose_push_response_list[0].body.decode()
response_data = json.loads(response_data)
if udict.get_by_path(response_data, 'resultInfo.message', '') != '批转成功!' or udict.get_by_path(
response_data, 'resultInfo.success') is not True:
raise PushException("批转请求发生错误.", dcm_dispose.flow_token, 3)
# 保存成功状态
dcm_dispose.status = 1
await dcm_dispose.async_save()
# 批转请求提交后,通知批转成功
await oa_result_notify.push_result_notify(
dcm_dispose.flow_token,
'批转成功',
1
)
# 如果需要发送短信,后台发送
if dcm_dispose.send_message in (1, '1'):
await dcm_send_sms.send_sms({
'actDefID': '45',
'partStr': '254,role,市受理员',
'itemType': 'transit',
'recID': dcm_task.rec_id,
'actID': dcm_task.act_id,
'actPropertyID': '7'
})
except PushException as e:
# 任何异常都意味着失败,通知 OA
echo_log(f'批转发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
# 保存失败状态
dcm_dispose.status = 0
await dcm_dispose.async_save()
# 批转发生异常,通知批转失败
await oa_result_notify.push_result_notify(
e.flow_token, f"{e}", e.return_code
)
except Exception as e:
# 其他异常都意味着失败
echo_log(f'批转发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
if __name__ == "__main__":
from paste.core import aio_pool
async def push():
dcm_task = await DcmTask(id=2059280827086409734).async_find_first()
task_dispose = await DcmDispose(id=2060189471659397120).async_find_first()
await push_dispose(task_dispose, dcm_task)
_runner = aio_pool.get_aio_runner()
_runner(push())
+235
View File
@@ -0,0 +1,235 @@
import asyncio
import datetime
import io
import json
import logging
from tornado.httpclient import HTTPResponse, HTTPRequest
import apps
import dock
from dock.dcm import dcm_api, dcm_push_conv_rollback, dcm_send_sms, dcm_push_upload
from dock.oa import oa_api_request, oa_result_notify, PushException
from models.dcm_rollback import DcmRollback
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.util import ufile, udict
from paste.web import requests
async def get_rollback_request(dcm_rollback: DcmRollback, dcm_task: DcmTask, media_ids: str, media_types: str,
add_num: int, extra_opinion: str):
"""
创建回退请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:param dcm_rollback: 回退对象
:param dcm_task: 工单对象
:param media_ids: 上传的附件 MediaID 字符串,多个用英文逗号分隔
:param media_types: 上传的附件的MediaType字符串,多个用英文逗号分隔
:param add_num: 上传的附件数
:param extra_opinion: 便民表单的extendInfo文本
:return: HTTPRequest 对象
"""
api_url = '/home/workflow/rollback'
body_dict = {
'actID': int(dcm_rollback.act_id),
'opinion': dcm_rollback.opinion if not extra_opinion else dcm_rollback.opinion + ' ' + extra_opinion,
'transInfo': dcm_rollback.trans_info,
'saveOldActFlag': dcm_rollback.save_old_act_flag,
'rollbackReasonID': dcm_rollback.rollback_reason_id,
'transListNum': 0,
'mediaIDs': media_ids,
'mediaUsage': '回退',
'relationTypeID': 10,
'relationID': dcm_task.rec_id,
'relationMainID': 45,
'relationSubID': dcm_task.act_id,
'tempUsage': 'rollback_tmp',
'mediaTypes': media_types,
'addNum': add_num
}
rollback_request = await dcm_api.new_api_request(api_url, body_dict)
return rollback_request
async def on_attachment_download_error(request: HTTPRequest, exc: Exception, retry_queue: asyncio.Queue = None):
"""
下载附件时的出错处理。
:param request: 请求对象
:param exc: 异常对象
:param retry_queue:
:return:
"""
retry = getattr(request, 'retry', 0)
max_retry = getattr(request, 'max_retry', 0)
if retry < max_retry - 1:
# 非最后一次尝试,不处理
return
message = f"下载回退附件时发生错误,下载地址:{request.url}"
echo_log(message)
echo_log(exc, logging.ERROR, is_log_exc=True)
# 保存异常
dcm_rollback: DcmRollback = getattr(request, 'dcm_rollback')
exc_list = getattr(dcm_rollback, 'exc_list')
exc_list.append(PushException(message, dcm_rollback.flow_token, 3))
async def done_attachment_download(response_list: list[HTTPResponse]):
"""
所有附件下载完成执行的处理程序。
:param response_list: 附件下载响应列表
:return: 返回附件字典
"""
# 遍历取出附件数据
date_str = datetime.datetime.now().strftime("%Y%m%d")
files_body: dict[str, io.IOBase] = {}
for i, response in enumerate(response_list):
_type = ufile.inspect_type(response.body)
if isinstance(_type, tuple):
_fn = f"CT{date_str}{i + 1:04d}.{_type[0]}"
elif isinstance(_type, str) and _type:
_fn = f"CT{date_str}{i + 1:04d}.{_type}"
else:
_fn = f"CT{date_str}{i + 1:04d}"
file_obj = io.BytesIO(response.body)
files_body[_fn] = file_obj
return files_body
async def after_push_rollback_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('回退请求成功.')
async def push_rollback(dcm_rollback: DcmRollback, dcm_task: DcmTask):
"""
推送回退请求。
:param dcm_rollback: 保存在数据库的回退对象
:param dcm_task: 待办工单
"""
try:
# 如果来源是12345热线,提交便民回退
extra_opinion = ''
if dcm_task.event_src_name == '12345热线':
conv_response_list = await dcm_push_conv_rollback.push_conv_rollback_request(dcm_rollback, dcm_task)
if conv_response_list:
conv_response = conv_response_list[0].body.decode()
conv_response = json.loads(conv_response)
extra_opinion = udict.get_by_path(conv_response, 'resultInfo.data.extendInfo', '')
echo_log(f"正在准备下载队列...")
setattr(dcm_rollback, 'exc_list', [])
files_body = {}
if dcm_rollback.attachments:
# 创建并填充下载请求队列
attachment_download_queue = asyncio.Queue()
attachment_id_list = dcm_rollback.attachments.split(',')
for attachment_id in attachment_id_list:
download_request = await oa_api_request.get_download_request(attachment_id)
setattr(download_request, 'dcm_rollback', dcm_rollback)
await attachment_download_queue.put(download_request)
# 启动下载,得到所有的附件
files_body: dict[str, io.IOBase] = await requests.async_concurrency(
attachment_download_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_done=done_attachment_download, on_error=on_attachment_download_error
)
# 读取异常有则报错
exc_list: list[PushException] = getattr(dcm_rollback, 'exc_list', [])
if exc_list:
message = ";".join([f"{exc}" for exc in exc_list])
raise PushException(message, dcm_rollback.flow_token, 3)
echo_log(f"附件下载完成,正在准备提交数字城管...")
# 仅生产环境真实提交,其他环境不实际提交
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
# 上传附件,同步得到 MediaIds
media_ids, media_types, add_num = await dcm_push_upload.push_upload(
dcm_task, dcm_push_upload.RollbackTmp, files_body
)
# 创建并填充数字城管请求队列
rollback_request = await get_rollback_request(
dcm_rollback, dcm_task, media_ids, media_types, add_num, extra_opinion
)
setattr(rollback_request, 'dcm_rollback', dcm_rollback)
rollback_push_queue = asyncio.Queue()
await rollback_push_queue.put(rollback_request)
# 提交数字城管-回退请求
rollback_push_response_list = await requests.async_concurrency(
rollback_push_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_push_rollback_request
)
# 检查是否推送成功,失败直接报错
if len(rollback_push_response_list) != 1:
raise PushException("回退请求发生错误.", dcm_rollback.flow_token, 3)
# 检查响应的内容,验证是否推送成功
response_data = rollback_push_response_list[0].body.decode()
response_data = json.loads(response_data)
if udict.get_by_path(response_data, 'resultInfo.message', '') != '回退成功!' or udict.get_by_path(
response_data, 'resultInfo.success') is not True:
raise PushException("回退请求发生错误.", dcm_rollback.flow_token, 3)
# 保存成功状态
dcm_rollback.status = 1
await dcm_rollback.async_save()
# 回退请求提交后,通知回退成功
await oa_result_notify.push_result_notify(
dcm_rollback.flow_token,
'回退成功',
1
)
# 如果需要发送短信,后台发送
if dcm_rollback.send_message in (1, '1'):
await dcm_send_sms.send_sms({
'actDefID': '45',
'partStr': '254,role,市受理员',
'itemType': 'rollback',
'recID': dcm_task.rec_id,
'actID': dcm_task.act_id,
'actPropertyID': '7'
})
except PushException as e:
# 任何异常都意味着失败,通知 OA
echo_log(f'回退发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
# 保存失败状态
dcm_rollback.status = 0
await dcm_rollback.async_save()
# 回退发生异常,通知回退失败
await oa_result_notify.push_result_notify(
e.flow_token, f"{e}", e.return_code
)
except Exception as e:
# 其他异常都意味着失败
echo_log(f'回退发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
if __name__ == "__main__":
from paste.core import aio_pool
async def push():
dcm_task = await DcmTask(id=2054174091237265413).async_find_first()
task_rollback = await DcmRollback(id=2058802518821048320).async_find_first()
await push_rollback(task_rollback, dcm_task)
_runner = aio_pool.get_aio_runner()
_runner(push())
+109
View File
@@ -0,0 +1,109 @@
import asyncio
import logging
from tornado.httpclient import HTTPResponse, HTTPRequest
import apps
import dock
from dock.dcm import dcm_api
from dock.oa import PushException, oa_result_notify
from models.dcm_stage_reply import DcmStageReply
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.web import requests
async def get_stage_reply_request(dcm_stage_reply: DcmStageReply, dcm_task: DcmTask):
"""
创建阶段回复请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:return: HTTPRequest 对象
"""
api_url = '/home/mis/convenient/stagereply'
body_dict = {
"recID": dcm_task.rec_id,
"actID": dcm_task.act_id,
"opinion": dcm_stage_reply.content,
"itemType": dcm_stage_reply.item_type,
}
return await dcm_api.new_api_request(api_url, body_dict)
async def after_push_stage_reply_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('阶段回复请求成功.')
async def push_stage_reply(dcm_stage_reply: DcmStageReply, dcm_task: DcmTask):
"""
推送阶段回复请求。
:param dcm_stage_reply: 保存在数据库的阶段回复对象
:param dcm_task: 待办工单
"""
try:
# 仅生产环境真实提交,其他环境不实际提交
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
# 创建并填充数字城管请求队列
stage_reply_request = await get_stage_reply_request(dcm_stage_reply, dcm_task)
setattr(stage_reply_request, 'dcm_stage_reply', dcm_stage_reply)
stage_reply_push_queue = asyncio.Queue()
await stage_reply_push_queue.put(stage_reply_request)
# 提交数字城管-阶段回复请求
stage_reply_push_response_list = await requests.async_concurrency(
stage_reply_push_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_push_stage_reply_request
)
# 检查是否推送成功,失败直接报错
if len(stage_reply_push_response_list) != 1:
raise PushException("阶段回复请求发生错误.", dcm_stage_reply.flow_token, 3)
# TODO: 这里要做实际的推送成功检查
# 保存成功状态
dcm_stage_reply.status = 1
await dcm_stage_reply.async_save()
# 阶段回复请求提交后,通知阶段回复成功
await oa_result_notify.push_result_notify(
dcm_stage_reply.flow_token,
'阶段回复成功',
1
)
except PushException as e:
# 任何异常都意味着失败,通知 OA
echo_log(f'阶段回复发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
# 保存失败状态
dcm_stage_reply.status = 0
await dcm_stage_reply.async_save()
# 阶段回复发生异常,通知阶段回复失败
await oa_result_notify.push_result_notify(
e.flow_token, f"{e}", e.return_code
)
except Exception as e:
# 其他异常都意味着失败,通知 OA
echo_log(f'阶段回复发生错误.', logging.ERROR)
echo_log(e, logging.ERROR, is_log_exc=True)
if __name__ == '__main__':
async def push():
reply = await DcmStageReply.async_find_by_id(2061285857339510784)
task = await DcmTask.async_find_by_id(2054174091270819843)
await push_stage_reply(reply, task)
from paste.core import aio_pool
_runner = aio_pool.get_aio_runner()
_runner(push())
+197
View File
@@ -0,0 +1,197 @@
"""
数字城管文件上传。
"""
import asyncio
import datetime
import io
import json
import logging
from typing import Optional
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api
from dock.oa import PushException
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.util import udict
from paste.web import requests
ApplyPostponeTmp = "apply_postpone_tmp"
ApplyRollbackTmp = "apply_rollback_tmp"
RollbackTmp = "rollback_tmp"
TransitTmp = "transit_tmp"
MediaUsage = [ApplyPostponeTmp, ApplyRollbackTmp, RollbackTmp, TransitTmp]
"""
可用媒体对象。
"""
async def get_upload_request(dcm_task: DcmTask, media_usage: str, files: dict[str, io.IOBase],
relation_sub_id: Optional[int] = None,
relation_main_id: Optional[int] = 45):
"""
创建文件上传请求。因数字城管服务器仅接受单文档上传,因此返回的是请求对象列表。
:param dcm_task: 待办工单
:param media_usage: 媒体使用
:param files: 要上传的文件数据
:param relation_sub_id: 可选参数
:param relation_main_id: 可选参数
:return: 上传请求列表
"""
api_url = f"/media/upload"
assert media_usage in MediaUsage, f"参数错误,须为:{MediaUsage} 之一."
request_list = []
_id = 0
for key, value in files.items():
request_body = {
"relationTypeID": 10,
"mediaUsage": media_usage,
"checkExist": True,
"relationID": dcm_task.rec_id,
"relationSubID": relation_sub_id if relation_sub_id is not None else dcm_task.act_id,
"relationMainID": relation_main_id,
"tempUsage": media_usage,
"id": f"WU_FILE_{_id}",
"name": key,
"type": 'image/png',
"lastModifiedDate": datetime.datetime.now().strftime("%a %b %d %Y %H:%M:%S %z").replace(
"+0800", "GMT+0800"
),
"size": len(value.read()),
key: value,
}
_id += 1
# 构造 API 请求
request = await dcm_api.new_api_request(api_url, request_body)
setattr(request, 'dcm_task', dcm_task)
setattr(request, 'media_usage', media_usage)
setattr(request, 'file_name', key)
request_list.append(request)
# 返回请求列表
return request_list
async def get_media_request(dcm_task: DcmTask, media_usage: str):
"""
创建读取 MediaID 请求。
:param dcm_task: 待办工单
:param media_usage: 媒体使用
:return: 上传请求列表
"""
api_url = f"/media/get"
assert media_usage in MediaUsage, f"参数错误,须为:{MediaUsage} 之一."
request_body = {
"relationTypeID": 10,
"mediaUsage": media_usage,
"relationID": dcm_task.rec_id,
"relationSubID": dcm_task.act_id,
"relationMainID": 45,
}
# 构造 API 请求
request = await dcm_api.new_api_request(api_url, request_body, method="GET")
setattr(request, 'dcm_task', dcm_task)
setattr(request, 'media_usage', media_usage)
return request
async def on_upload_error(request: HTTPRequest, exc: Exception, retry_queue: asyncio.Queue = None):
"""
下载附件时的出错处理。
:param request: 请求对象
:param exc: 异常对象
:param retry_queue:
:return:
"""
retry = getattr(request, 'retry', 0)
max_retry = getattr(request, 'max_retry', 0)
if retry < max_retry - 1:
# 非最后一次尝试,不处理
return
dcm_task: DcmTask = getattr(request, 'dcm_task')
media_usage = getattr(request, 'media_usage')
file_name = getattr(request, 'file_name')
message = f'工单ID{dcm_task.id},类型为:{media_usage},文件名为:{file_name} 的附件上传失败.'
echo_log(message)
echo_log(exc, logging.ERROR, is_log_exc=True)
# 保存异常
exc_list = getattr(dcm_task, 'exc_list')
exc_list.append(PushException(message))
async def after_upload_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
dcm_task: DcmTask = getattr(response.request, 'dcm_task')
media_usage = getattr(response.request, 'media_usage')
file_name = getattr(response.request, 'file_name')
echo_log(response.body.decode())
echo_log(f'工单ID{dcm_task.id},类型为:{media_usage},文件名为:{file_name} 的附件上传成功.')
async def push_upload(dcm_task: DcmTask, media_usage: str, files: dict[str, io.IOBase],
relation_sub_id: Optional[int] = None,
relation_main_id: Optional[int] = 45):
"""
向数字城管上传文件。
:param dcm_task: 待办工单
:param media_usage: 媒体使用
:param files: 要上传的文件数据
:param relation_sub_id: 可选参数
:param relation_main_id: 可选参数
:return:
"""
echo_log(f"正在准备上传队列...")
request_list = await get_upload_request(dcm_task, media_usage, files, relation_sub_id, relation_main_id)
upload_request_queue = asyncio.Queue()
for req in request_list:
await upload_request_queue.put(req)
# 并发提交上传请求
await requests.async_concurrency(
upload_request_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_upload_request, on_error=on_upload_error
)
# 上传后,读取文件 MediaID
media_request = await get_media_request(dcm_task, media_usage)
media_request_queue = asyncio.Queue()
await media_request_queue.put(media_request)
media_response_list: list[HTTPResponse] = await requests.async_concurrency(
media_request_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT
)
# 从响应列表中取得媒体ID和媒体类型ID
media_id_list: list = []
media_type_list: list = []
media_num = 0
if media_response_list:
response = media_response_list[0]
response_body = response.body.decode()
response_data = json.loads(response_body)
media_list = udict.get_by_path(response_data, 'data.mediaList')
media_num = len(media_list)
for media in media_list:
media_id_list.append(f"{media.get('mediaID')}")
media_type_list.append(media.get('mediaType'))
# 用英文逗号拼接后返回
return ",".join(media_id_list), ",".join(media_type_list), media_num
if __name__ == "__main__":
from paste.core import aio_pool
_runner = aio_pool.get_aio_runner()
+160
View File
@@ -0,0 +1,160 @@
"""
数据抓取模块。
"""
import asyncio
import logging
from typing import Optional, Union
from sqlalchemy import select, desc
import dock
from dock.dcm.dcm_scrape_operation import get_operation_request, after_operation_request
from dock.dcm.dcm_scrape_task import get_task_request, after_task_request
from dock.dcm.dcm_scrape_attachment import get_attachment_request, after_attachment_request
from dock.dcm.dcm_scrape_process_info import get_process_info_request, after_process_info_request
from dock.dcm.dcm_scrape_form_data import get_form_data_request, after_form_data_request
from dock.dcm.dcm_scrape_more_info import get_more_info_request, after_more_info_request
from dock.dcm.dcm_scrape_extend_info import get_extend_info_request, after_extend_info_request
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.web import requests
async def fetch_dcm_task(fetch_size: int = 60, task_id: Optional[Union[str, int]] = None):
"""
抓取待办数据及其明细数据。
:param fetch_size: 读取多少任务进行明细抓取
:param task_id: 待办任务 ID 可选
"""
echo_log(f"开始抓取待办数据...")
task_request = await get_task_request(num_per_page=fetch_size)
request_queue = asyncio.Queue()
await request_queue.put(task_request)
await requests.async_concurrency(request_queue, retry=dock.MAX_RETRY_COUNT, after_request=after_task_request)
echo_log(f"待办数据抓取完成...")
# 读取任务数据,以便能对最新数据抓取详细数据
query = select(
DcmTask.id, DcmTask.rec_id, DcmTask.act_id, DcmTask.task_num, DcmTask.other_task_num,
).order_by(
desc(DcmTask.act_id)
)
if task_id:
if isinstance(task_id, list):
query = query.where(DcmTask.id.in_(task_id))
echo_log(f"开始抓取待办列表:{task_id} 的详细数据...")
else:
query = query.where(DcmTask.id == task_id)
echo_log(f"开始抓取待办:{task_id} 的详细数据...")
else:
echo_log(f"开始抓取前 {fetch_size} 条待办的详细数据...")
query = query.limit(fetch_size)
task_df = await DcmTask.query_as_df(query)
# 构建请求队列
operation_queue = asyncio.Queue()
attachment_queue = asyncio.Queue()
process_info_queue = asyncio.Queue()
form_data_queue = asyncio.Queue()
more_info_queue = asyncio.Queue()
extend_info_queue = asyncio.Queue()
# 向队列中填充请求对象
echo_log(f"正在准备请求队列...")
for _h, _row in task_df.iterrows():
dcm_task_id = _row.get(DcmTask.id.key)
rec_id = int(_row.get(DcmTask.rec_id.key))
act_id = int(_row.get(DcmTask.act_id.key))
task_num = f"{_row.get(DcmTask.task_num.key)}"
other_task_num = f"{_row.get(DcmTask.other_task_num.key)}"
_operation_req = await get_operation_request(rec_id, act_id, task_num, other_task_num)
echo_log(_operation_req.url)
setattr(_operation_req, "dcm_task_id", dcm_task_id)
setattr(_operation_req, "rec_id", rec_id)
await operation_queue.put(_operation_req)
_attachment_req = await get_attachment_request(rec_id)
echo_log(_attachment_req.url)
setattr(_attachment_req, "dcm_task_id", dcm_task_id)
setattr(_attachment_req, "rec_id", rec_id)
await attachment_queue.put(_attachment_req)
_process_info_req = await get_process_info_request(rec_id)
echo_log(_process_info_req.url)
setattr(_process_info_req, "dcm_task_id", dcm_task_id)
setattr(_process_info_req, "rec_id", rec_id)
await process_info_queue.put(_process_info_req)
_form_data_req = await get_form_data_request(rec_id, act_id)
echo_log(_form_data_req.url)
setattr(_form_data_req, "dcm_task_id", dcm_task_id)
setattr(_form_data_req, "rec_id", rec_id)
await form_data_queue.put(_form_data_req)
_more_info_req = await get_more_info_request(rec_id)
echo_log(_more_info_req.url)
setattr(_more_info_req, "dcm_task_id", dcm_task_id)
setattr(_more_info_req, "rec_id", rec_id)
await more_info_queue.put(_more_info_req)
_extend_info_req = await get_extend_info_request(rec_id)
echo_log(_extend_info_req.url)
setattr(_extend_info_req, "dcm_task_id", dcm_task_id)
setattr(_extend_info_req, "rec_id", rec_id)
await extend_info_queue.put(_extend_info_req)
_count = (operation_queue.qsize()+attachment_queue.qsize()+process_info_queue.qsize()
+form_data_queue.qsize()+more_info_queue.qsize()+extend_info_queue.qsize())
echo_log(f"可用操作请求:{operation_queue.qsize()};附件请求:{attachment_queue.qsize()};处理过程请求:{process_info_queue.qsize()}")
echo_log(f"详细数据请求:{form_data_queue.qsize()};更多信息请求:{more_info_queue.qsize()};扩展信息请求:{extend_info_queue.qsize()}")
echo_log(f"共计:{_count} 个.")
echo_log(f"抓取待办详细数据...")
try:
tasks = [
requests.async_concurrency(
operation_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_operation_request
),
requests.async_concurrency(
attachment_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_attachment_request
),
requests.async_concurrency(
process_info_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_process_info_request
),
requests.async_concurrency(
form_data_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_form_data_request
),
requests.async_concurrency(
more_info_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_more_info_request
),
requests.async_concurrency(
extend_info_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_extend_info_request
)
]
await asyncio.gather(*tasks)
except Exception as e:
echo_log(f"抓取任务异常: {e}", level=logging.ERROR, is_log_exc=True)
raise
async def fetch_single_dcm_task(dcm_task: DcmTask):
"""
基于任务抓取指定某条待办数据及其明细数据。
:param dcm_task: 任务号
"""
await fetch_dcm_task(task_id=dcm_task.id)
if __name__ == "__main__":
from paste.core import aio_pool
_runner = aio_pool.get_aio_runner()
_runner(fetch_dcm_task(30))
+86
View File
@@ -0,0 +1,86 @@
import asyncio
import json
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api
from models.dcm_task import DcmTask
from paste.util import udict
from paste.web import requests
async def get_allow_postpone_request(
rec_id: int, act_id: int, task_num: str, other_task_num: str,
task_list_id: int = 600058,
task_list_name: str = "部门待办栏"
):
"""
获取 DCM 企业待办的是否允许申请延期。
向 DCM 的是否允许申请延期接口发送 GET 请求,获取与指定关系 ID 和类型 ID 关联的可用菜单信息。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
rec_id (int): 记录 ID
act_id (int): 任务 ID
task_num (str): 任务号
other_task_num (str): 第三方任务号
task_list_id: 任务列表类型 ID,默认为企业待办:600058
task_list_name: 任务列表名称,默认为:部门待办栏
"""
api_url = f"/home/workflow/wfitemtypecheck?itemType=postpone"
request_body = {
"recID": rec_id,
"actID": act_id,
"tasknum": task_num,
"otherTaskNum": other_task_num,
"taskListID": task_list_id,
"taskListName": task_list_name,
"recDispNum": "",
"menuName": "assign",
"menuDisplayName": "办理",
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'POST', timeout=30)
async def after_allow_postpone_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
setattr(response.request, "response_data", response_data)
async def fetch_allow_postpone(dcm_task: DcmTask):
"""
抓取是否允许申请延期。
:param dcm_task: 数字城管待办工单
:return:
"""
# 取得请求对象
list_menu_request = await get_allow_postpone_request(
dcm_task.rec_id, dcm_task.act_id, dcm_task.task_num, dcm_task.other_task_num
)
request_queue = asyncio.Queue()
await request_queue.put(list_menu_request)
await requests.async_concurrency(
request_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_allow_postpone_request
)
response_data = getattr(list_menu_request, "response_data")
success: bool = udict.get_by_path(response_data, 'resultInfo.success')
message: str = udict.get_by_path(response_data, 'resultInfo.message')
return success, message
if __name__ == "__main__":
from paste.core import aio_pool
async def test(dcm_task_id):
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
await fetch_allow_postpone(dcm_task)
_runner = aio_pool.get_aio_runner()
_runner(test(2054174091287597056))
+52
View File
@@ -0,0 +1,52 @@
import asyncio
import json
import pandas as pd
import models
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.dcm import dcm_api
from paste.util import udict
from paste.core.logging import echo_log
from models.dcm_task_attachment import DcmTaskAttachment
async def get_attachment_request(relation_id: int, relation_type_id: int = 1):
"""
获取 DCM 企业待办的附件列表。
向 DCM 的附件查询接口发送 GET 请求,获取与指定关系 ID 和类型 ID 关联的附件信息。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
relation_id (int): 关联记录的 ID,例如任务 ID。
relation_type_id (int): 关联类型 ID,默认为 1(任务类型)。
"""
api_url = f"/home/mis/attachrec/getattach"
request_body = {
"relationID": relation_id,
"relationTypeID": relation_type_id,
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET')
async def after_attachment_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
list_data = udict.get_by_path(response_data, 'resultInfo.data.mediaList')
attachment_df = pd.DataFrame(list_data)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in DcmTaskAttachment.FieldMapping.items()}
mapped_df = attachment_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
dcm_task_id = getattr(response.request, 'dcm_task_id')
rec_id = getattr(response.request, 'rec_id')
mapped_df[DcmTaskAttachment.dcm_task_id.key] = dcm_task_id
mapped_df[DcmTaskAttachment.rec_id.key] = rec_id
# 筛选数据状态
_created, _updated = await DcmTaskAttachment.save_batch(mapped_df)
echo_log(f"成功创建企业待办 {rec_id} 的附件:{_created}条,更新:{_updated}条.")
if retry_queue:
echo_log(f"企业待办附件重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+73
View File
@@ -0,0 +1,73 @@
import asyncio
import json
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api, dcm_scrape_convenient_form
from models.dcm_task import DcmTask
from paste.util import udict
from paste.web import requests
async def get_convenient_request(act_id: int, other_task_num: str = "transit"):
"""
获取 DCM 企业待办的便民批转。
向 DCM 的便民批转接口发送 GET 请求,获取与指定关系 ID 和类型 ID 关联的可用菜单信息。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
act_id (int): 任务 ID
other_task_num (str): 项目类型
"""
api_url = f"/home/workflow/gettranstreewithoutconfig"
request_body = {
"actID": act_id,
"itemType": other_task_num,
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET', timeout=30)
async def after_convenient_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
setattr(response.request, "response_data", response_data)
async def fetch_form(dcm_task: DcmTask):
"""
抓取便民批转。
:param dcm_task: 数字城管待办工单
:return:
"""
# 取得请求对象
list_menu_request = await get_convenient_request(dcm_task.act_id)
request_queue = asyncio.Queue()
await request_queue.put(list_menu_request)
await requests.async_concurrency(
request_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_convenient_request
)
response_data = getattr(list_menu_request, "response_data")
form: dict = udict.get_by_path(response_data, 'resultInfo.data.form')
if form:
components = await dcm_scrape_convenient_form.fetch_form_components(
dcm_task, form.get('formID', dcm_scrape_convenient_form.DisposeFormId)
)
form['components'] = components
return form
if __name__ == "__main__":
from paste.core import aio_pool
async def test(dcm_task_id):
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
await fetch_form(dcm_task)
_runner = aio_pool.get_aio_runner()
_runner(test(2054174091287597056))
+73
View File
@@ -0,0 +1,73 @@
import asyncio
import json
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api, dcm_scrape_convenient_form
from models.dcm_task import DcmTask
from paste.util import udict
from paste.web import requests
async def get_convenient_request(act_id: int, item_type: str = "rollback"):
"""
获取 DCM 企业待办的便民回退。
向 DCM 的便民回退接口发送 GET 请求,获取与指定关系 ID 和类型 ID 关联的可用菜单信息。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
act_id (int): 任务 ID
item_type (str): 项目类型
"""
api_url = f"/home/workflow/gettranstree"
request_body = {
"actID": act_id,
"itemType": item_type,
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET', timeout=30)
async def after_convenient_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
setattr(response.request, "response_data", response_data)
async def fetch_form(dcm_task: DcmTask):
"""
抓取便民回退。
:param dcm_task: 数字城管待办工单
:return:
"""
# 取得请求对象
list_menu_request = await get_convenient_request(dcm_task.act_id)
request_queue = asyncio.Queue()
await request_queue.put(list_menu_request)
await requests.async_concurrency(
request_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_convenient_request
)
response_data = getattr(list_menu_request, "response_data")
form: dict = udict.get_by_path(response_data, 'resultInfo.data.form')
if form:
components = await dcm_scrape_convenient_form.fetch_form_components(
dcm_task, form.get('formID', dcm_scrape_convenient_form.RollbackFormId)
)
form['components'] = components
return form
if __name__ == "__main__":
from paste.core import aio_pool
async def test(dcm_task_id):
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
await fetch_form(dcm_task)
_runner = aio_pool.get_aio_runner()
_runner(test(2054174091287597056))
+85
View File
@@ -0,0 +1,85 @@
import asyncio
import json
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api
from models.dcm_task import DcmTask
from paste.util import udict
from paste.web import requests
DisposeFormId = 377
DisposeFormName = '便民批转'
RollbackFormId = 352
RollbackFormName = '便民回退'
FormIdMap = {
DisposeFormId: DisposeFormName,
RollbackFormId: RollbackFormName,
}
async def get_convenient_form_request(rec_id: int, act_id: int, form_id: int = DisposeFormId):
"""
获取 DCM 企业待办的便民表单组件。
向 DCM 的便民表单组件接口发送 GET 请求,获取与指定关系 ID 和类型 ID 关联的可用菜单信息。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
rec_id (int): 记录 ID
act_id (int): 任务 ID
form_id (str): 项目类型,允许输入:377、352,默认:377,表示读取:"便民批转"表单,为空时读取:"便民回退"表单
"""
api_url = f"/home/form/formpreview/getforminfo"
request_body = {
"formID": form_id if form_id in tuple(FormIdMap.keys()) else DisposeFormId,
"formName": FormIdMap.get(form_id) if form_id in tuple(FormIdMap.keys()) else DisposeFormName,
"param": {
"recID": rec_id,
"actID": act_id,
},
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'POST', timeout=30)
async def after_convenient_form_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
setattr(response.request, "response_data", response_data)
async def fetch_form_components(dcm_task: DcmTask, form_id: int = DisposeFormId):
"""
抓取便民表单组件。
:param dcm_task: 数字城管待办工单ID
:param form_id: 项目类型,允许输入:377、352,默认:377,表示读取:"便民批转"表单,为空时读取:"便民回退"表单
:return:
"""
# 取得请求对象
list_menu_request = await get_convenient_form_request(dcm_task.rec_id, dcm_task.act_id, form_id)
request_queue = asyncio.Queue()
await request_queue.put(list_menu_request)
await requests.async_concurrency(
request_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_convenient_form_request
)
response_data = getattr(list_menu_request, "response_data")
component_list: list[dict] = udict.get_by_path(response_data, 'resultInfo.data.form.componentList')
return component_list
if __name__ == "__main__":
from paste.core import aio_pool
async def test(dcm_task_id, form_id):
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
await fetch_form_components(dcm_task, form_id)
_runner = aio_pool.get_aio_runner()
_runner(test(2054174091287597056, '352'))
+50
View File
@@ -0,0 +1,50 @@
import asyncio
import json
import pandas as pd
import models
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.dcm import dcm_api
from paste.util import udict
from paste.core.logging import echo_log
from models.dcm_task_extend_info import DcmTaskExtendedInfo
async def get_extend_info_request(rec_id: int):
"""
获取 DCM 企业代办扩展信息数据。
向 DCM 的企业代办扩展信息数据接口发送 GET 请求,获取指定记录ID的扩展信息数据。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
rec_id (int): 关联类型 ID,默认为 1(任务类型)。
"""
api_url = f"/home/mis/rec/getsubtypeexvalue"
request_body = {
"recID": rec_id
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET')
async def after_extend_info_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
list_data = udict.get_by_path(response_data, 'resultInfo.data.list')
data_df = pd.DataFrame(list_data)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in DcmTaskExtendedInfo.FieldMapping.items()}
mapped_df = data_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
dcm_task_id = getattr(response.request, 'dcm_task_id')
rec_id = getattr(response.request, 'rec_id')
mapped_df[DcmTaskExtendedInfo.dcm_task_id.key] = dcm_task_id
mapped_df[DcmTaskExtendedInfo.rec_id.key] = rec_id
# 筛选数据状态
_created, _updated = await DcmTaskExtendedInfo.save_batch(mapped_df)
echo_log(f"成功创建企业待办 {rec_id} 的扩展信息:{_created}条,更新:{_updated}条.")
if retry_queue:
echo_log(f"企业待办扩展信息重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+61
View File
@@ -0,0 +1,61 @@
import asyncio
import json
import pandas as pd
import models
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.dcm import dcm_api
from paste.util import udict
from paste.core.logging import echo_log
from models.dcm_task_form_datum import DcmTaskFormDatum
async def get_form_data_request(rec_id: int, act_id: int, form_id: int = 356):
"""
获取 DCM 企业待办表单预览数据。
向 DCM 的企业待办表单预览数据接口发送 GET 请求,获取与指定关系 ID 和类型 ID 关联的媒体信息。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
rec_id (int): 关联类型 ID,默认为 1(任务类型)。
act_id (int): 关联类型 ID,默认为 1(任务类型)。
form_id (int): 表单 ID。
"""
api_url = f"/home/form/formpreview/getformdata"
# 注意:以下字典加入 body 前必须先编码,否则服务端可能无法解析
_params = {
"recID": rec_id,
"actID": act_id,
}
request_body = {
"formID": form_id,
"param": json.dumps(_params, separators=(',', ':')),
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET')
async def after_form_data_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body or '{}')
list_data_546 = udict.get_by_path(response_data, 'resultInfo.data.formTableData.546') or {}
list_data_588 = udict.get_by_path(response_data, 'resultInfo.data.formTableData.588') or {}
list_data_588.pop('rec_id', None)
list_data = {**list_data_546, **list_data_588}
form_data_df = pd.DataFrame([list_data])
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in DcmTaskFormDatum.FieldMapping.items()}
mapped_df = form_data_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
dcm_task_id = getattr(response.request, 'dcm_task_id')
rec_id = getattr(response.request, 'rec_id')
mapped_df[DcmTaskFormDatum.dcm_task_id.key] = dcm_task_id
mapped_df[DcmTaskFormDatum.rec_id.key] = rec_id
# 筛选数据状态
_created, _updated = await DcmTaskFormDatum.save_batch(mapped_df)
echo_log(f"成功创建企业待办 {rec_id} 的表单:{_created}条,更新:{_updated}条.")
if retry_queue:
echo_log(f"企业待办表单重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+50
View File
@@ -0,0 +1,50 @@
import asyncio
import json
import pandas as pd
import models
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.dcm import dcm_api
from paste.util import udict
from paste.core.logging import echo_log
from models.dcm_task_more_info import DcmTaskMoreInfo
async def get_more_info_request(rec_id: int):
"""
获取 DCM 企业代办更多信息数据。
向 DCM 的企业代办更多信息数据接口发送 GET 请求,获取指定记录ID的更多信息数据。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
rec_id (int): 关联类型 ID,默认为 1(任务类型)。
"""
api_url = f"/home/workflow/getrecmsginfo"
request_body = {
"recID": rec_id
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET')
async def after_more_info_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
list_data = udict.get_by_path(response_data, 'resultInfo.data.processInfo')
data_df = pd.DataFrame(list_data)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in DcmTaskMoreInfo.FieldMapping.items()}
mapped_df = data_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
dcm_task_id = getattr(response.request, 'dcm_task_id')
rec_id = getattr(response.request, 'rec_id')
mapped_df[DcmTaskMoreInfo.dcm_task_id.key] = dcm_task_id
mapped_df[DcmTaskMoreInfo.rec_id.key] = rec_id
# 筛选数据状态
_created, _updated = await DcmTaskMoreInfo.save_batch(mapped_df)
echo_log(f"成功创建企业待办 {rec_id} 的更多信息:{_created}条,更新:{_updated}条.")
if retry_queue:
echo_log(f"企业待办更多信息重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+119
View File
@@ -0,0 +1,119 @@
import asyncio
import json
from sqlalchemy import select
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api
from models.dcm_task import DcmTask
from paste.core.logging import echo_log
from paste.util import udict
from paste.web import requests
async def get_operation_request(
rec_id: int, act_id: int, task_num: str, other_task_num: str,
task_list_id: int = 600058,
task_list_name: str = "部门待办栏"
):
"""
获取 DCM 企业待办的可用菜单列表。
向 DCM 的可用菜单列表接口发送 GET 请求,获取与指定关系 ID 和类型 ID 关联的可用菜单信息。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
rec_id (int): 记录 ID
act_id (int): 任务 ID
task_num (str): 任务号
other_task_num (str): 第三方任务号
task_list_id: 任务列表类型 ID,默认为企业待办:600058
task_list_name: 任务列表名称,默认为:部门待办栏
"""
api_url = f"/home/workflow/assign"
request_body = {
"recID": rec_id,
"actID": act_id,
"tasknum": task_num,
"otherTaskNum": other_task_num,
"taskListID": task_list_id,
"taskListName": task_list_name,
"recDispNum": "",
"menuName": "assign",
"menuDisplayName": "办理",
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET', timeout=30)
async def after_operation_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
menus: list[dict] = udict.get_by_path(response_data, 'resultInfo.data.menuData.menus')
operation: list[str] = []
if menus is None:
menus = []
for menu in menus:
if menu.get('menuName') != 'unassign' and menu.get('visible', False) and menu.get('displayProperty', 0) & 2 == 2:
# 排除撤销办理,和不显示的菜单,仅返回可用的操作菜单
operation.append(menu.get('displayName'))
setattr(response.request, "response_data", response_data)
setattr(response.request, "operation", response_data)
dcm_task_id = getattr(response.request, 'dcm_task_id')
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
dcm_task.operation = ','.join(operation)
await dcm_task.async_save()
rec_id = getattr(response.request, 'rec_id')
echo_log(f"成功更新企业待办 {rec_id} 的可用操作:{dcm_task.operation}.")
if retry_queue:
echo_log(f"企业待办可用操作重试队列中有:{retry_queue.qsize()} 个请求在等待.")
async def fetch_operation(dcm_task: DcmTask):
"""
抓取列表菜单可用操作。
:param dcm_task: 数字城管待办工单
:return:
"""
# 取得请求对象
operation_request = await get_operation_request(
dcm_task.rec_id, dcm_task.act_id, dcm_task.task_num, dcm_task.other_task_num
)
setattr(operation_request, "dcm_task_id", dcm_task.id)
request_queue = asyncio.Queue()
await request_queue.put(operation_request)
await requests.async_concurrency(
request_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_operation_request
)
operation = getattr(operation_request, "operation")
return operation
if __name__ == "__main__":
from paste.core import aio_pool
async def test(dcm_task_id):
dcm_task = await DcmTask(id=dcm_task_id).async_find_first()
assert dcm_task, f"未找到待办工单,工单ID{dcm_task_id}"
await fetch_operation(dcm_task)
async def test_all():
dcm_task_list: list[DcmTask] = await DcmTask.query_all(select(DcmTask))
queue = asyncio.Queue()
for dcm_task in dcm_task_list:
request = await get_operation_request(
dcm_task.rec_id, dcm_task.act_id, dcm_task.task_num, dcm_task.other_task_num
)
setattr(request, "dcm_task_id", dcm_task.id)
setattr(request, "rec_id", dcm_task.rec_id)
await queue.put(request)
await requests.async_concurrency(queue, after_request=after_operation_request)
_runner = aio_pool.get_aio_runner()
# _runner(test(2054174091304374276))
_runner(test_all())
+54
View File
@@ -0,0 +1,54 @@
import asyncio
import json
import pandas as pd
import models
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.dcm import dcm_api
from paste.util import udict
from paste.core.logging import echo_log
from models.dcm_task_process_info import DcmTaskProcessInfo
async def get_process_info_request(relation_id: int, process_type: str = 'full', show_assign_flag: int = 0):
"""
获取 DCM 企业待办处理经过信息。
向 DCM 的任务处理经过信息接口发送 GET 请求,获取指定任务的流程信息(如审批流、节点等)。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 数据。
Args:
relation_id (int): 关联记录的 ID,例如任务 ID。
process_type (str): 流程信息类型,默认为 'full'(完整流程)。
show_assign_flag (int): 是否显示分配人信息,0 表示不显示,非 0 表示显示。
"""
api_url = f"/home/workflow/getrecprocessinfo"
request_body = {
"recID": relation_id,
"processType": process_type,
"showAssignFlag": show_assign_flag,
}
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body, 'GET')
async def after_process_info_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
list_data = udict.get_by_path(response_data, 'resultInfo.data.processInfo')
process_info_df = pd.DataFrame(list_data)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in DcmTaskProcessInfo.FieldMapping.items()}
mapped_df = process_info_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
dcm_task_id = getattr(response.request, 'dcm_task_id')
rec_id = getattr(response.request, 'rec_id')
mapped_df[DcmTaskProcessInfo.dcm_task_id.key] = dcm_task_id
mapped_df[DcmTaskProcessInfo.rec_id.key] = rec_id
# 筛选数据状态
_created, _updated = await DcmTaskProcessInfo.save_batch(mapped_df)
echo_log(f"成功创建企业待办 {rec_id} 的经过:{_created}条,更新:{_updated}条.")
if retry_queue:
echo_log(f"企业待办经过重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+67
View File
@@ -0,0 +1,67 @@
import asyncio
import json
import pandas as pd
import models
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.dcm import dcm_api
from paste.util import udict
from paste.core.logging import echo_log
from models.dcm_task import DcmTask
async def get_task_request(task_list_id: int = 600058, current_page: int = 1, num_per_page: int = 200,
sort_field_id: int = -1, sort_type: str = '', only_data_flag: bool = False,
search_num: str = None):
"""
获取 DCM 任务列表数据。
通过 POST 请求向 DCM 的任务列表接口提交表单数据,获取任务分页数据。
自动注入有效的 Cookie(如 JSESSIONID)至请求头,并解析返回的 JSON 响应。
Args:
task_list_id (int): 任务列表类型 ID,默认为企业待办:600058
current_page (int): 当前页码。
num_per_page (int): 每页显示数据量,默认 200。
sort_field_id (int): 排序字段 ID,默认不排序为 -1。
sort_type (str): 排序类型,默认无排序字段,排序类型为空。
only_data_flag (bool): 仅数据标志,默认为:False。
search_num(str): 可选的任务号关键词,默认搜索时不传此参数。
"""
api_url = f"/home/bizbase/tasklist/gethumantasklistdata"
request_body = {
"taskListID": task_list_id,
"currentPage": current_page,
"numPerPage": num_per_page,
"sortFieldID": sort_field_id,
"sortType": sort_type,
"onlyDataFlag": only_data_flag,
}
if search_num is not None:
request_body["searchNum"] = search_num
# 构造 API 请求
return await dcm_api.new_api_request(api_url, request_body)
async def after_task_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
任务请求响应后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
response_body = response.body.decode()
response_data = json.loads(response_body)
list_data = udict.get_by_path(response_data, 'resultInfo.data.listDataSet.listData')
task_df = pd.DataFrame(list_data)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in DcmTask.FieldMapping.items()}
mapped_df = task_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
# 筛选数据状态
_created, _updated = await DcmTask.save_batch(mapped_df)
echo_log(f"成功创建企业待办:{_created}条,更新:{_updated}条.")
if retry_queue:
echo_log(f"企业待办重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+115
View File
@@ -0,0 +1,115 @@
"""
安全模块。
"""
import asyncio
import json
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api
from models.token import TokenModel
from paste.core import config
from paste.core.logging import echo_log
from paste.util import udict
from paste.web import requests
async def login():
"""
登录 DCM 系统并获取认证 Cookie 和响应数据。
流程:
1. 访问 `/main.htm` 获取服务端下发的初始 Cookie(如 JSESSIONID)。
2. 使用该 Cookie 构造请求头,向 `/login/validpassword` 发送 POST 请求。
3. 请求体包含用户名、密码、浏览器及操作系统信息(从配置和随机生成获取)。
4. 验证响应中 `resultInfo.success` 字段,若为 False 则抛出断言错误。
Args:
无参数。
Returns:
tuple: 包含两个元素的元组:
- str: 请求头中的 Cookie 字符串(如 "JSESSIONID=abc; ..."
- dict: DCM 接口返回的完整 JSON 响应数据
Raises:
AssertionError: 登录失败(`resultInfo.success` 为 False
ValueError: 响应体非合法 JSON
HTTPError: 网络请求失败(由 `async_request` 抛出)
"""
home_url = f"{dcm_api.ApiUrl}/main.htm"
login_url = f"{dcm_api.ApiUrl}/login/validpassword"
# 获取初始 Cookie
initial_cookies = await dock.scrape_cookies(home_url)
cookie_header = "; ".join([f"{k}={v}" for k, v in initial_cookies.items()])
echo_log(cookie_header)
# 构建扩展头
user_agent, browser_ver, os_name = dock.get_random_user_agent()
extra_headers = {
'Cookie': cookie_header,
'User-Agent': user_agent,
}
# 构造请求
request_body = {
"u": config.get_config("dock.dcm.account.u"),
"p": config.get_config("dock.dcm.account.p"),
"ip": "",
"browserVersion": browser_ver,
"osVersion": os_name,
"validCode": "",
"validWay": 0,
}
# 构造请求对象
request = dock.new_http_request(
url=login_url,
body=request_body,
method='POST',
timeout=dock.DEFAULT_TIMEOUT,
use_form=True,
extra_headers=extra_headers,
)
setattr(request, 'cookie_header', cookie_header)
queue = asyncio.Queue()
await queue.put(request)
await requests.async_concurrency(
queue, con_count=1, retry=dock.MAX_RETRY_COUNT,
after_request=after_login
)
async def after_login(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
response_body = response.body.decode()
response_data = json.loads(response_body)
success = udict.get_by_path(response_data, 'resultInfo.success', False)
if success:
cookie_header = getattr(response.request, 'cookie_header')
await TokenModel.refresh(platform='数字城管', token=cookie_header)
echo_log(f"成功刷新数字城管登录令牌.")
else:
echo_log(f"数字城管登录失败,无法刷新令牌,响应:{response_body}")
if retry_queue:
echo_log(f"登录重试队列中有:{retry_queue.qsize()} 个请求在等待.")
return response_data
async def get_cookies(platform: str = '数字城管'):
"""
取得可用 Cookies。目前固定,后期改为从数据库读取。
:param platform: 要查询的平台,默认是:数字城管
:return: Cookies 字符串
"""
_token = await TokenModel.find_by_platform(platform)
return _token.token
if __name__ == "__main__":
from paste.core import aio_pool
_runner = aio_pool.get_aio_runner()
_runner(login())
+51
View File
@@ -0,0 +1,51 @@
import asyncio
from tornado.httpclient import HTTPResponse, HTTPRequest
import dock
from dock.dcm import dcm_api
from paste.core.logging import echo_log
from paste.web import requests
import apps
async def get_send_sms_request(body):
"""
创建发送短信的请求对象。方法仅创建请求对象,并未实际提交请求,具体由调度方法处理。
:param body: 短信参数字典
:return: HTTPRequest 对象
"""
api_url = '/home/sms/sendwfsms'
sms_request = await dcm_api.new_api_request(api_url, body)
return sms_request
async def after_send_sms_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
提交数字城管后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
echo_log(response.body.decode())
echo_log('发送短信成功')
async def send_sms(body):
"""
调用数字城管的发送短信接口
:param body: 短信参数字典
"""
echo_log('正在准备发送短信...')
sms_request = await get_send_sms_request(body)
sms_queue = asyncio.Queue()
await sms_queue.put(sms_request)
if apps.get_active_env() in ('dev', '', None):
echo_log(f"非生产环境,不实际提交.")
return
await requests.async_concurrency(
sms_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=after_send_sms_request
)
+3
View File
@@ -0,0 +1,3 @@
"""
市12345对接模块。
"""
Binary file not shown.
Binary file not shown.
+65
View File
@@ -0,0 +1,65 @@
"""
市12345对接 API 基础功能。
"""
from tornado.httpclient import AsyncHTTPClient
import dock
from paste.core import config
ApiUrl = "http://2.46.12.176:8091/sz12345"
"""
对接 API 根目录。
"""
ProxyConfig = config.get_config('dock.govc.proxy')
"""
代理服务器配置。
"""
if ProxyConfig and ProxyConfig.get('proxy_host', None) and ProxyConfig.get('proxy_port', None):
# 切换到底层实现,以便代理服务器生效
AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
async def new_api_request(api_url: str, request_body: dict, method: str = 'POST',
timeout: float = dock.DEFAULT_TIMEOUT, use_form: bool = True, headers: dict = None):
"""
构造一个 API 请求对象
:param api_url: API 地址,以斜杠开头的 URI 地址,非完整 URL
:param request_body: 请求体,即所有请求参数
:param method: 请求提交方式
:param timeout: 超时时长
:param use_form: 是否使用表单(Form)方式提交
:param headers: 头数据,最高优先级
:return: HTTPRequest 对象
"""
# Cookie
from dock.govc import govc_security
_, cookie_header = await govc_security.get_cookies()
# 构建扩展头
user_agent, browser_ver, os_name = dock.get_random_user_agent()
extra_headers = {
'Cookie': cookie_header,
'EPTOKEN': dock.get_cookie_value(cookie_header, 'EPTOKEN'),
'Host': '2.46.12.176:8091',
'Origin': 'http://2.46.12.176:8091',
'Referer': 'http://2.46.12.176:8091/sz12345/bmfw/bmfwlogin/login',
'User-Agent': user_agent,
}
if headers is not None:
extra_headers = {**extra_headers, **headers}
# 构造请求对象
request = dock.new_http_request(
url=f"{ApiUrl}{api_url}",
body=request_body,
method=method,
timeout=timeout,
use_form=use_form,
extra_headers=extra_headers,
** ProxyConfig
)
return request
+94
View File
@@ -0,0 +1,94 @@
"""
数据抓取模块。
"""
import asyncio
from typing import Optional, Union
from sqlalchemy import select, desc
import dock
from dock.govc import govc_scrape_dept_feedback, govc_scrape_return_visit, govc_scrape_finish_info, govc_scrape_order
from models.govc_task import GovcTask
from paste.core.logging import echo_log
from paste.web import requests
async def fetch_govc_task(fetch_size: int = 60, task_id: Optional[Union[str, int]] = None):
"""
抓取待办数据及其明细数据。
:param fetch_size: 读取多少任务进行明细抓取
:param task_id: 可选的指定的工单id
"""
echo_log(f"开始抓取待办数据...")
task_request = await govc_scrape_order.get_task_request(
fetch_size=fetch_size
)
request_queue = asyncio.Queue()
await request_queue.put(task_request)
await requests.async_concurrency(
request_queue, retry=dock.MAX_RETRY_COUNT,
after_request=govc_scrape_order.after_task_request
)
echo_log(f"待办数据抓取完成...")
# 读取任务数据,以便能对最新数据抓取详细数据
query = select(
GovcTask.id, GovcTask.pvi_guid, GovcTask.c_guid
).order_by(
desc(GovcTask.id)
)
if task_id:
if isinstance(task_id, list):
query = query.where(GovcTask.id.in_(task_id))
echo_log(f"开始抓取待办列表:{task_id} 的详细数据...")
else:
query = query.where(GovcTask.id == task_id)
echo_log(f"开始抓取待办:{task_id} 的详细数据...")
else:
echo_log(f"开始抓取前 {fetch_size} 条待办的详细数据...")
query = query.limit(fetch_size)
task_df = await GovcTask.query_as_df(query)
# 构建请求队列
feedback_queue = asyncio.Queue()
result_info_queue = asyncio.Queue()
finish_info_queue = asyncio.Queue()
# 向队列中填充请求对象
echo_log(f"正在准备请求队列...")
for _h, _row in task_df.iterrows():
_feedback_request = await govc_scrape_dept_feedback.get_feedback_request(_row.get(GovcTask.pvi_guid.key),
_row.get(GovcTask.c_guid.key))
setattr(_feedback_request, 'task_id', _row.get(GovcTask.id.key))
await feedback_queue.put(_feedback_request)
_result_info_request = await govc_scrape_return_visit.get_return_visit_request(_row.get(GovcTask.pvi_guid.key),
_row.get(GovcTask.c_guid.key))
setattr(_result_info_request, 'task_id', _row.get(GovcTask.id.key))
await result_info_queue.put(_result_info_request)
_finish_info_request = await govc_scrape_finish_info.get_finish_info_request(_row.get(GovcTask.pvi_guid.key),
_row.get(GovcTask.c_guid.key))
setattr(_finish_info_request, 'task_id', _row.get(GovcTask.id.key))
await finish_info_queue.put(_finish_info_request)
echo_log(f"抓取待办详细数据...")
tasks = [
requests.async_concurrency(
feedback_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=govc_scrape_dept_feedback.after_feedback_request
),
requests.async_concurrency(
result_info_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=govc_scrape_result_info.after_result_info_request
),
requests.async_concurrency(
finish_info_queue, con_count=dock.CONCURRENCY_COUNT, retry=dock.MAX_RETRY_COUNT,
after_request=govc_scrape_finish_info.after_finish_info_request)
]
await asyncio.gather(*tasks)
if __name__ == "__main__":
from paste.core import aio_pool
_runner = aio_pool.get_aio_runner()
_runner(fetch_govc_task(10))
+59
View File
@@ -0,0 +1,59 @@
import asyncio
import json
import pandas as pd
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.govc import govc_api
import models
from models.govc_task_contact import GovcTaskContact
from paste.util import udict
from paste.core.logging import echo_log
async def get_contact_request(pviguid: str, cguid: str):
"""
获取市12345的工单的联系信息,请求响应是单条工单的数据
:param pviguid: 工单列表请求返回的pviguid
:param cguid: 工单列表请求返回的cguid
"""
api_url = '/rest/sztaskworkordercommonrest/getContactInformation'
headers = {
'Referer': f'{govc_api.ApiUrl}/rest/sztaskworkordercommonrest/getContactInformation'
}
request_body = {
"ProcessVersionInstanceGuid": pviguid,
'caseguid': cguid,
'yearflag': 'undefined'
}
# 构造 API 请求
return await govc_api.new_api_request(api_url, request_body, headers=headers)
async def after_contact_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
任务请求响应后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
response_body = response.body.decode()
response_data = json.loads(response_body)
contact_list = udict.get_by_path(response_data, 'params.linklist')
if contact_list:
mapped_df = pd.DataFrame(contact_list)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in GovcTaskContact.FieldMapping.items()}
mapped_df = mapped_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
task_id = getattr(response.request, GovcTaskContact.task_id.key)
mapped_df[GovcTaskContact.task_id.key] = task_id
# 筛选数据状态
_created, _updated = await GovcTaskContact.save_batch(mapped_df)
echo_log(f"成功创建联系信息:{_created}条,更新:{_updated}条.")
else:
echo_log('未获取到联系信息')
if retry_queue:
echo_log(f"联系信息重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+59
View File
@@ -0,0 +1,59 @@
import asyncio
import json
import pandas as pd
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.govc import govc_api
import models
from models.govc_task_delay import GovcTaskDelay
from paste.util import udict
from paste.core.logging import echo_log
async def get_delay_request(pviguid: str, cguid: str):
"""
获取市12345的工单的延迟信息,请求响应是单条工单的数据
:param pviguid: 工单列表请求返回的pviguid
:param cguid: 工单列表请求返回的cguid
"""
api_url = '/rest/sztaskhandlerest/getDelayInfo'
headers = {
'Referer': f'{govc_api.ApiUrl}/rest/sztaskhandlerest/getDelayInfo'
}
request_body = {
"ProcessVersionInstanceGuid": pviguid,
'caseguid': cguid,
'yearflag': 'undefined'
}
# 构造 API 请求
return await govc_api.new_api_request(api_url, request_body, headers=headers)
async def after_delay_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
任务请求响应后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
response_body = response.body.decode()
response_data = json.loads(response_body)
delay_list = udict.get_by_path(response_data, 'params.table')
if delay_list:
mapped_df = pd.DataFrame(delay_list)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in GovcTaskDelay.FieldMapping.items()}
mapped_df = mapped_df.rename(columns=forward_mapping)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
task_id = getattr(response.request, GovcTaskDelay.task_id.key)
mapped_df[GovcTaskDelay.task_id.key] = task_id
# 筛选数据状态
_created, _updated = await GovcTaskDelay.save_batch(mapped_df)
echo_log(f"成功创建延迟信息:{_created}条,更新:{_updated}条.")
else:
echo_log('未获取到延迟信息')
if retry_queue:
echo_log(f"延迟信息重试队列中有:{retry_queue.qsize()} 个请求在等待.")
+66
View File
@@ -0,0 +1,66 @@
import asyncio
import json
import pandas as pd
from tornado.httpclient import HTTPResponse, HTTPRequest
from dock.govc import govc_api
import models
from models.govc_task_department_feedback import GovcTaskDeptFeedback
from paste.util import udict
from paste.core.logging import echo_log
async def get_feedback_request(pviguid: str, cguid: str):
"""
获取市12345的工单的部门处置信息,请求响应是单条工单的数据
:param pviguid: 工单列表请求返回的pviguid
:param cguid: 工单列表请求返回的cguid
"""
api_url = '/rest/sztaskworkordercommonrest/getDeptfeedback'
headers = {
'Referer': f'{govc_api.ApiUrl}/rest/sztaskworkordercommonrest/getDeptfeedback'
}
request_body = {
"ProcessVersionInstanceGuid": pviguid,
'caseguid': cguid,
'yearflag': 'undefined'
}
# 构造 API 请求
return await govc_api.new_api_request(api_url, request_body, headers=headers)
async def after_feedback_request(response: HTTPResponse, retry_queue: asyncio.Queue[HTTPRequest]):
"""
任务请求响应后的处理程序。
:param response: 响应对象
:param retry_queue: 重试队列
"""
response_body = response.body.decode()
response_data = json.loads(response_body)
feedback_list = udict.get_by_path(response_data, 'params.feedbackresult')
if feedback_list:
mapped_df = pd.DataFrame(feedback_list)
# 更换映射方向,用于将源数据列名改为与数据库表对应
forward_mapping = {dict_f: table_f for table_f, dict_f in GovcTaskDeptFeedback.FieldMapping.items()}
mapped_df = mapped_df.rename(columns=forward_mapping)
# 把字典、列表改为字符串
mapped_df[GovcTaskDeptFeedback.zxhf_info.key] = mapped_df[GovcTaskDeptFeedback.zxhf_info.key].apply(
lambda x: json.dumps(x, ensure_ascii=False) if isinstance(x, (list, dict)) else x
)
mapped_df[GovcTaskDeptFeedback.back_info.key] = mapped_df[GovcTaskDeptFeedback.back_info.key].apply(
lambda x: json.dumps(x, ensure_ascii=False) if isinstance(x, (list, dict)) else x
)
# 这里把空数据都换成 None,以便存入数据库时是 null
mapped_df.replace(models.EmptyInDF + models.EmptyDatetimeInDF, None, inplace=True)
task_id = getattr(response.request, GovcTaskDeptFeedback.task_id.key)
mapped_df[GovcTaskDeptFeedback.task_id.key] = task_id
# 筛选数据状态
_created, _updated = await GovcTaskDeptFeedback.save_batch(mapped_df)
echo_log(f"成功创建部门处置信息:{_created}条,更新:{_updated}条.")
else:
echo_log('未获取到部门处置信息')
if retry_queue:
echo_log(f"部门处置信息重试队列中有:{retry_queue.qsize()} 个请求在等待.")

Some files were not shown because too many files have changed in this diff Show More