{"repo":"Rainynitesky/kaipanla-data-parser","free":true,"listed":false,"github":"https://github.com/Rainynitesky/kaipanla-data-parser","clone":"git clone https://github.com/Rainynitesky/kaipanla-data-parser.git","description":"开盘啦 App 数据抓取与解析工具 - 批量抓取板块/个股数据，Token自动更新，mitmproxy流量拦截","language":"Python","stars":59,"topics":["crawler","data-parser","mitmproxy","python","stock-data","kaipanla"],"license":"MIT","category":"scrapers-browser-automation","readme_excerpt":"开盘啦数据解析项目 项目目标 抓取开盘啦App(com.aiyu.kaipanla)的 板块行情数据 及 板块-个股关联数据 ，输出为CSV。 板块列表通过 RealRankingInfo API 动态分页获取（不再依赖静态 kaipanla-plates-267.json 文件），数量随市场变化。 --- 1. 核心发现（已确认，勿重复验证） 1.1 267板块列表走Socket不走HTTP - PlateTCConfig HTTP接口只返回58个行业板块 - 完整267板块（含概念板块）通过 TCP Socket + TLS + Protobuf 传输 - Socket服务器: 124.71.166.244:14000 , 1.94.128.64:14000 , 60.204.212.123:14000 - IP获取接口: getsockip.longhuvip.com/getIPList?UserID=...&Token=... (HTTP, 已验证可用) - 板块列表已通过 RealRankingInfo API 动态获取，无需静态映射文件 1.2 Socket协议栈 - DEX中发现 getSocketAESKey / getHttpAESKey → Socket通信有AES加密层 - libsockSign.so 负责签名计算，AuthReq.signature字段需要此库生成 1.3 App加固 - libzxprotect.so 加固框架，导致： - Frida Java.perform 报 \"Java is not defined\" — Java层hook完全不可用 - Frida Native hook（send/recv/SSL write/BIO write）全部0调用 - classes.dex是壳，真正代码被加密 - frida-dexdump运行时脱壳成功 ：41/54个DEX已dump 1.4 App网络行为 - 不强制SSL pinning — mitmproxy无需Frida bypass即可解密HTTPS - Socket连接绕过系统HTTP代理 — 不走CONNECT隧道，直连服务器 - iptables DNAT在MuMu模拟器上 不生效 - 可行的Socket拦截方案 : mitmproxy劫持getIPList响应，把IP改为10.0.2.2:14000 --- 2. 已尝试且失败的方案（勿重复尝试） 方案 结果 原因 ------ ------ ------ Frida Java.perform hook ❌ libzxprotect.so加固导致Java VM不可达 Frida Native hook send/recv ❌ 加固替换libc函数指针 Frida Native hook SSL write/BIO write ❌ App用内嵌BoringSSL iptables DNAT重定向Socket ❌ MuMu模拟器QEMU用户模式网络不经过内核netfilter Python直连Socket发AuthReq ❌ AES加密层+签名缺失，服务器关闭连接 GetConceptJXBKw23 HTTP接口 ❌ 接口存在但返回空List jadx反编译 ❌ classes.dex是壳无法静态分析 --- 3. 已成功的方案 方案 结果 产出 ------ ------ ------ mitmproxy抓包HTTPS API ✅ captures/ 800+个JSON frida-dexdump运","default_branch":null,"files":null,"tree":[],"storefront":"/r/Rainynitesky","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Rainynitesky/kaipanla-data-parser/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}