Why use AI and Big Data in Finance?
- explosion of data
- 能够更快嗅到 insights 的企业会获得巨大优势.
- 对冲基金用 ML 提前捕捉信号
- cost saving & efficiency
- 合规、放贷、欺诈检测自动化
- risk management
- 提前识别市场异常或违约
金融数据包括什么？
- Market Data
- Customer Data
- Alternative Data
- Regulatory Data
caution: data governance.
Big Data in Finance
- Volume
- Velocity: real-time stream data
- Variety: structured + unstructured data
- Veracity: noice in data
Storage
非关系类数据储存：
- Data lake
- NoSQL
- Cloud
Processing
(low latency)
- Batch processing
- Real-time
- ETL pipeline
Cloud and AI
cloud AI platforms
- Access to advanced ML frameworks without in-house infrastructure.
- Scalability for demand spikes
- Lower upfront costs for smaller FinTechs.
劣势：
- Data sovereignty and regulatory restrictions (cross-border data transfers)
- Vendor lock-in with major cloud providers
Barriers
- Data quality issues: Missing, noisy, or biased datasets reduce model reliability.
- Black-box models: Lack of interpretability conflicts with regulatory requirements.
- Talent gap: Demand for data scientists in finance far exceeds supply.
- Integration challenges: Legacy systems in banks make it difficult to implement advanced AI solutions.
- Ethical risks: Bias in lending decisions, privacy concerns, job displacement.