xinyouhe/photo-live
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(app): path-precise duplicate matching; shared file stat; union test

Browse Source 
Review follow-ups on 0fa23e0 (Task 2), applied as one commit:

- Path-precise dupPaths (#2, user-visible): DuplicateMatch gains
  sourcePath (local match fills it from the matched pick; unified
  server branch from picks[hit.index].path). Both pages' duplicate
  dialogs derive dupPaths from duplicates' sourcePath instead of
  basename matching, so same-basename siblings (different dir/bytes)
  are no longer mis-flagged — "跳过重复" no longer drops the
  non-duplicate sibling. Stale basename-derivation comment removed.
- Single shared stat (#3): files are stat'd once in
  checkDuplicatesUnified (_statPicks) and the picks feed both the
  local half (extracted _matchLocalHistory; public checkDuplicates
  folded — it had no remaining callers) and the server request.
  Halves stat IO for large batches (was ~2x).
- Union test (#1): new test/features/uploads/enqueue_test.dart pins
  that a file hit by BOTH local and server history yields ONE
  DuplicateMatch with the local uploadedAt (server's echoed time does
  not override). Merge dedup is now keyed by sourcePath
  (putIfAbsent, local first).

Tests: +1 enqueue union test (real-async chain inside one runAsync
window; sync file/dir setup — fake-async body must not await real
IO), +1 album_detail same-basename sibling test (skip-duplicates
enqueues exactly the non-hit sibling, distinguished by bytes since
sourceFilePath becomes the preparer's staging copy).

Gates: flutter analyze clean; full flutter test 429/429 green.

Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
浅念
2026-09-28 20:22:37 +08:00
parent 0fa23e015e
commit ac37794d43
5 changed files with 289 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/mobile/lib/features/uploads/enqueue.dart
+68 -56
View File
@@ -8,8 +8,10 @@
/// 内部有界并行复制，默认并发 3，防连拍 IO 风暴）。
///
/// P0-1 层 1（UI06 分析 v2）：入队前**本地查重**——同账号 + 同相册 +
/// state=uploaded + filename+bytes 匹配 → 返回 [DuplicateCheck]，由
/// state=uploaded + filename+bytes 匹配 → 返回 [DuplicateMatch]，由
/// 调用方决定弹窗确认或跳过。批量场景汇总后单次弹窗（非逐张）。
/// 本地匹配经统一入口 [checkDuplicatesUnified] 驱动（原独立入口
/// checkDuplicates 已折叠为私有半场，仓库内无其他调用方）。
///
/// Task 2（查重前置到入队前·统一入口）：入队前查重升级为
/// [checkDuplicatesUnified]——本机 TaskStore 历史 ∪ 服务器历史
@@ -99,11 +101,15 @@ typedef DuplicateDialogChoice
/// 查重结果（单张）：匹配到的已上传任务摘要。
class DuplicateMatch {
const DuplicateMatch({
required this.sourcePath,
required this.filename,
required this.bytes,
required this.uploadedAt,
});
/// 本批照片的完整源路径：弹窗/入队按路径精确派生重复项——同 basename
/// 不同目录/扩展名的兄弟文件不误伤（Task 2 评审 #2）。
final String sourcePath;
final String filename;
final int bytes;
final DateTime uploadedAt;
@@ -127,30 +133,38 @@ class EnqueueResult {
final int skipped;
}
/// 入队前查重：查 TaskStore 同账号 + 同相册 + state=uploaded，
/// filename + bytes 匹配的记录。
///
/// 返回 null = 无重复（可直接入队）；非 null = 有疑似重复（调用方弹窗）。
Future<List<DuplicateMatch>?> checkDuplicates(
WidgetRef ref, {
required String accountId,
required String albumId,
required List<String> sourcePaths,
}) async {
if (sourcePaths.isEmpty) return null;
/// 本批照片的 stat 结果（路径 + 派生 filename + 字节数；层 1/层 2 查重
/// 共用——文件只 stat 一次，Task 2 评审 #3）。
typedef _PickStat = ({String path, String filename, int bytes});
// 收集本次选择的 filename + bytes。
final picks = <({String filename, int bytes})>[];
/// stat 本批文件（[checkDuplicatesUnified] 专用；读不到的文件跳过，
/// 入队时准备器会报错）。
Future<List<_PickStat>> _statPicks(List<String> sourcePaths) async {
final picks = <_PickStat>[];
for (final path in sourcePaths) {
try {
final file = File(path);
picks.add((filename: p.basename(path), bytes: await file.length()));
picks.add((
path: path,
filename: p.basename(path),
bytes: await file.length(),
));
} on Object {
// 读不到的文件不参与查重（入队时准备器会报错）。
}
}
if (picks.isEmpty) return null;
return picks;
}
/// 层 1 本机匹配：TaskStore 同账号 + 同相册 + state=completed 的任务与
/// 本批 stat 结果做 filename + bytes 匹配（原公共 checkDuplicates 的
/// 内核，折叠为统一入口的私有半场——仓库内已无其他调用方）。
Future<List<DuplicateMatch>> _matchLocalHistory(
WidgetRef ref, {
required String accountId,
required String albumId,
required List<_PickStat> picks,
}) async {
// 查同相册已完成的任务。
final store = ref.read(taskStoreProvider);
final completed = await store.listByAccount(
@@ -166,6 +180,7 @@ Future<List<DuplicateMatch>?> checkDuplicates(
if (task.filename == pick.filename && task.bytes == pick.bytes) {
matches.add(
DuplicateMatch(
sourcePath: pick.path,
filename: pick.filename,
bytes: pick.bytes,
uploadedAt: task.createdAt,
@@ -175,8 +190,7 @@ Future<List<DuplicateMatch>?> checkDuplicates(
}
}
}
return matches.isEmpty ? null : matches;
return matches;
}
/// 统一查重（P0-1 层 1+层 2 前置，Task 2）：本机 TaskStore 历史 ∪
@@ -186,8 +200,10 @@ Future<List<DuplicateMatch>?> checkDuplicates(
/// 服务器预检失败（网络/权限/任意异常）→ 静默降级本机-only（不阻塞
/// 入队；服务端 init 409 仍是兜底）。返回 null = 无重复。
///
/// 服务器命中按全局下标映射回本批文件（stat 本批取 filename，与
/// [checkDuplicates] 同模式），弹窗/强传标记的 filename 派生由此生效。
/// 本批文件只 stat 一次（[_statPicks]，两层共用）；服务器命中按全局
/// 下标映射回 stat 结果（filename/sourcePath 由此生效）；合并按
/// sourcePath 去重——同一文件双命中只呈现一条，本机命中在前（本地
/// uploadedAt 优先，Task 2 评审 #1）。
Future<List<DuplicateMatch>?> checkDuplicatesUnified(
WidgetRef ref, {
required String accountId,
@@ -196,58 +212,54 @@ Future<List<DuplicateMatch>?> checkDuplicatesUnified(
}) async {
if (sourcePaths.isEmpty) return null;
// stat 一次，层 1/层 2 共用。
final picks = await _statPicks(sourcePaths);
if (picks.isEmpty) return null;
// 层 1：本机 TaskStore 历史（filename+bytes 匹配）。
final local =
await checkDuplicates(
ref,
accountId: accountId,
albumId: albumId,
sourcePaths: sourcePaths,
) ??
const <DuplicateMatch>[];
final local = await _matchLocalHistory(
ref,
accountId: accountId,
albumId: albumId,
picks: picks,
);
// 层 2：服务器历史（bytes 命中，逐请求条目一条）。
final picks = <({String filename, int bytes})>[];
for (final path in sourcePaths) {
try {
final file = File(path);
picks.add((filename: p.basename(path), bytes: await file.length()));
} on Object {
// 读不到的文件不参与查重（入队时准备器会报错）。
}
}
// 层 2：服务器历史（bytes 命中，逐请求条目一条；失败静默降级）。
final serverHits = <UploadDuplicateHit>[];
if (picks.isNotEmpty) {
try {
serverHits.addAll(
await ref
.read(apiClientProvider)
.checkUploadDuplicates(albumId, picks),
);
} on Object {
// 静默降级本机-only：预检是体验优化而非闸门，失败不阻塞入队
//（服务端 init 409 仍是兜底）。
}
try {
serverHits.addAll(
await ref
.read(apiClientProvider)
.checkUploadDuplicates(
albumId,
[
for (final pick in picks)
(filename: pick.filename, bytes: pick.bytes),
],
),
);
} on Object {
// 静默降级本机-only：预检是体验优化而非闸门，失败不阻塞入队
//（服务端 init 409 仍是兜底）。
}
// 合并去重（filename+bytes；本机命中在前——同一重复保留先见者）。
final seen = <String>{};
final merged = <DuplicateMatch>[];
// 合并去重（sourcePath 唯一标识一张本批照片；putIfAbsent 保留先见
// 者——本机在前，故同一重复的 uploadedAt 取本地历史）。
final byPath = <String, DuplicateMatch>{};
for (final match in [
...local,
for (final hit in serverHits)
if (hit.index >= 0 && hit.index < picks.length)
DuplicateMatch(
sourcePath: picks[hit.index].path,
filename: picks[hit.index].filename,
bytes: hit.bytes,
uploadedAt: hit.uploadedAt,
),
]) {
if (seen.add('${match.filename}\u0000${match.bytes}')) {
merged.add(match);
}
byPath.putIfAbsent(match.sourcePath, () => match);
}
return merged.isEmpty ? null : merged;
return byPath.isEmpty ? null : byPath.values.toList();
}
/// 批量入队：通知权限单次请求 + 逐张准备入队。
apps/mobile/lib/pages/album_detail_page.dart
+3 -4
View File
@@ -388,10 +388,9 @@ class _AlbumDetailPageState extends ConsumerState<AlbumDetailPage> {
List<DuplicateMatch> duplicates,
List<String> allPaths,
) async {
final dupFilenames = duplicates.map((d) => d.filename).toSet();
final dupPaths = allPaths
.where((p) => dupFilenames.contains(p.split('/').last))
.toSet();
// 重复项按完整源路径精确派生（同 basename 不同目录/扩展名的兄弟
// 文件不误伤，Task 2 评审 #2）。
final dupPaths = duplicates.map((d) => d.sourcePath).toSet();
final nonDupCount = allPaths.length - dupPaths.length;
final choice = await showDialog<String>(
apps/mobile/lib/pages/capture_page.dart
+3 -5
View File
@@ -506,11 +506,9 @@ class _CapturePageState extends ConsumerState<CapturePage>
List<DuplicateMatch> duplicates,
List<String> allPaths,
) async {
// 找出与 duplicates 匹配的路径（filename 一致即视为重复项）。
final dupFilenames = duplicates.map((d) => d.filename).toSet();
final dupPaths = allPaths
.where((p) => dupFilenames.contains(p.split('/').last))
.toSet();
// 重复项按完整源路径精确派生（同 basename 不同目录/扩展名的兄弟
// 文件不误伤，Task 2 评审 #2）。
final dupPaths = duplicates.map((d) => d.sourcePath).toSet();
final nonDupCount = allPaths.length - dupPaths.length;
final choice = await showDialog<String>(
apps/mobile/test/features/uploads/enqueue_test.dart
+155
View File
@@ -0,0 +1,155 @@
/// checkDuplicatesUnified 单元测试（Task 2 评审 #1/#3）。
///
/// 页级测试覆盖弹窗三态；本文件钉住合并语义：同一文件被本机 TaskStore
/// 与服务器历史同时命中 → 单条结果（不重复呈现）且本地 uploadedAt 优先。
/// 经 Consumer 捕获 WidgetRef 驱动真实函数——drift/文件 stat 均真实异步，
/// 整体跑在 runAsync（真实事件循环）窗口内（同页级测试的 real-io 模式；
/// fake-async 区内不做真实异步 IO）。
library;
import 'dart:io';
import 'package:dio/dio.dart';
import 'package:flutter/widgets.dart' show SizedBox;
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:photo_live/core/api_client.dart';
import 'package:photo_live/core/session_store.dart';
import 'package:photo_live/features/uploads/enqueue.dart';
import 'package:photo_live/features/uploads/task_state.dart';
import 'package:photo_live/features/uploads/task_store.dart';
import '../../helpers/fakes.dart';
/// JPEG 魔数最小字节（同页级测试）。
const List<int> kJpgBytes = <int>[
0xFF, 0xD8, 0xFF, 0xE0, 0x00, 0x10, 0x4A, 0x46, 0x49, 0x46, 0x00,
];
void main() {
testWidgets('本机∪服务器：同文件双命中 → 单条结果 + sourcePath + 本地时间优先',
(tester) async {
final session = FakeSessionStore()
..session = Session(
token: 'tok-1',
expiresAt: DateTime.utc(2099, 1, 1),
account: 'alice',
);
final store = TaskStore.memory();
final tmp = Directory.systemTemp.createTempSync('enqueue-test-');
addTearDown(() async {
await store.close();
try {
tmp.deleteSync(recursive: true);
} on Object {
// Windows 句柄残留容忍（同页级测试）。
}
});
// 层 2 命中源：服务器预检命中 pick1（index 0；uploadedAt 早于本地）。
final adapter = FakeAdapter(
(o) => (
200,
{
'duplicates': [
{
'index': 0,
'bytes': kJpgBytes.length,
'uploadedAt': '2020-01-01T00:00:00Z',
},
],
},
),
);
final client = ApiClient(
sessionStore: session,
dio: Dio(BaseOptions(baseUrl: 'http://test.local'))
..httpClientAdapter = adapter,
);
late WidgetRef capturedRef;
await tester.pumpWidget(
ProviderScope(
overrides: [
...pageTestOverrides(
session: session,
taskStore: store,
client: client,
),
],
child: Consumer(
builder: (context, ref, _) {
capturedRef = ref;
return const SizedBox();
},
),
),
);
// 本批照片（同步写盘；pick2 字节不同 = 两层的非命中）。
final pick1 = File('${tmp.path}/pick1.jpg')..writeAsBytesSync(kJpgBytes);
final pick2 = File('${tmp.path}/pick2.jpg')
..writeAsBytesSync([...kJpgBytes, 0x00]);
// 层 1 命中源 + 统一查重：真实异步链整体在真实事件循环内完成。
final seededAt = DateTime.utc(2026, 9, 20, 12);
late final DateTime seedCreatedAt;
late List<DuplicateMatch>? result;
await tester.runAsync(() async {
final seed = await store.create(
UploadTaskDraft(
id: 'task-seed',
accountId: 'alice',
albumId: 'alb-1',
clientTaskId: 'ct-seed',
sourceFilePath: '${tmp.path}/seed.jpg',
filename: 'pick1.jpg',
bytes: kJpgBytes.length,
contentType: 'image/jpeg',
),
now: seededAt,
);
for (final state in [
UploadTaskState.queued,
UploadTaskState.uploading,
UploadTaskState.uploadedConfirming,
UploadTaskState.completed,
]) {
await store.updateState(seed.id, state);
}
seedCreatedAt = seed.createdAt;
result = await checkDuplicatesUnified(
capturedRef,
accountId: 'alice',
albumId: 'alb-1',
sourcePaths: [pick1.path, pick2.path],
);
});
expect(result, isNotNull, reason: '双命中至少一条');
final matches = result!;
expect(
matches, hasLength(1), reason: '同一文件本地+服务器双命中合并为单条（不重复呈现）');
expect(
matches.single.sourcePath,
pick1.path,
reason: '重复项携带完整源路径（路径精确派生 dupPaths 的依据）',
);
expect(
matches.single.uploadedAt,
seedCreatedAt,
reason: '本地历史时间优先（服务器回显 2020 不覆盖本地 2026-09-20）',
);
expect(
adapter.requests,
hasLength(1),
reason: '服务器预检发起（≤500 项单请求）',
);
expect(
(adapter.requests.single.body as Map<String, dynamic>)['items'],
hasLength(2),
reason: '两张均送预检（含本地已命中的那张——union 语义不做短路）',
);
});
}
apps/mobile/test/pages/album_detail_page_test.dart
+60
View File
@@ -599,6 +599,66 @@ void main() {
expect(find.text('可能已上传过'), findsNothing);
expect(find.text('Added 1 photo(s) to the upload queue'), findsOneWidget);
});
testWidgets('同 basename 兄弟文件：仅命中那张被标记/跳过（评审 #2 路径精确）',
(tester) async {
// 批内两个同名文件（不同目录、不同字节）：仅 dirA 那张服务器命中。
// basename 派生会把两张都误标为重复（「跳过重复」丢非重复项）；
// 路径精确派生只标记命中那张。
final adapter = detailAdapter(
duplicateCheck: (
200,
{
'duplicates': [
{
'index': 0,
'bytes': kJpgBytes.length,
'uploadedAt': '2026-09-25T08:00:00Z',
},
],
},
),
);
// 同步建目录：fake-async 区内不做真实异步 IO（同 enqueue_test 教训）。
final dirA = Directory('${tmp.path}/a')..createSync();
final dirB = Directory('${tmp.path}/b')..createSync();
final hit = File('${dirA.path}/IMG_SAME.jpg')
..writeAsBytesSync(kJpgBytes);
final other = File('${dirB.path}/IMG_SAME.jpg')
..writeAsBytesSync([...kJpgBytes, 0x00]);
galleryPaths.addAll([hit.path, other.path]);
await pumpDetailForUpload(tester, adapter: adapter);
final button = find.widgetWithText(OutlinedButton, 'Pick from gallery');
await tester.runAsync(() async {
(button.evaluate().single.widget as OutlinedButton).onPressed!();
await Future<void>.delayed(const Duration(milliseconds: 300));
});
await tester.pump(const Duration(milliseconds: 300));
expect(find.text('可能已上传过'), findsOneWidget);
expect(find.textContaining('1 张照片可能已上传到该相册'), findsOneWidget,
reason: '仅命中那张计为重复（同名兄弟不误伤）');
expect(find.text('跳过重复，上传 1 张'), findsOneWidget);
// 「跳过重复」：仅未命中的同名兄弟入队，命中那张被跳过。
await tester.tap(find.text('跳过重复，上传 1 张'));
await tester.pump(const Duration(milliseconds: 100));
await tester.runAsync(() async {
await Future<void>.delayed(const Duration(milliseconds: 300));
});
await tester.pumpAndSettle();
expect(find.textContaining('1 张已入队，1 张重复已跳过'), findsOneWidget);
final tasks = await taskStore.listByAccount('alice');
expect(tasks, hasLength(1), reason: '命中那张被跳过，仅同名兄弟入队');
expect(
tasks.single.bytes,
kJpgBytes.length + 1,
reason: '入队的是未命中那张（字节不同可区分；sourceFilePath 已是'
'准备器复制路径，原始选图路径不再保留）；命中那张（同 basename）'
'被精确跳过',
);
});
});
}