Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ When a developer says *use the nova_dev skill to setup and build the app* or

Nova is a Flutter-based mobile AI assistant that:

- Runs AI models entirely on-device using `flutter_gemma`
- Runs AI models entirely on-device using `flutter_edge_ai` (Flutter Edge AI)
- Supports multiple inference engines (LiteRtLm, MediaPipe)
- Provides voice input, screen capture, and tool execution capabilities
- Uses RAG (Retrieval-Augmented Generation) with local memory
Expand All @@ -57,9 +57,9 @@ Nova is a Flutter-based mobile AI assistant that:

| Package | Version | Purpose |
|---------|---------|---------|
| `flutter_gemma` | ^1.11.0 | Core Gemma inference engine |
| `flutter_gemma_litertlm` | ^1.8.4 | LiteRT LM inference backend |
| `flutter_gemma_mediapipe` | ^1.0.7 | MediaPipe inference backend |
| `flutter_edge_ai` | ^2.0.0 | Core on-device inference engine |
| `flutter_edge_ai_litertlm` | ^1.8.7 | LiteRT LM inference backend |
| `flutter_edge_ai_mediapipe` | ^1.0.9 | MediaPipe inference backend |

### Storage and Persistence

Expand Down Expand Up @@ -409,7 +409,7 @@ import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart';

// 3. Package imports
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:shared_preferences/shared_preferences.dart';

// 4. Project imports
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Changed

- Migrate on-device inference from discontinued `flutter_gemma` packages to Flutter Edge AI 2.0 (`flutter_edge_ai`, `flutter_edge_ai_litertlm`, `flutter_edge_ai_mediapipe`, `flutter_edge_ai_speech`). Raise minimum Flutter SDK to 3.44.

## [0.4.8] - 2026-09-22

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Also available to other agents via [`.agents/skills/nova-dev`](.agents/skills/no

## Features

- **On-device AI** — Gemma / LiteRT via `flutter_gemma`
- **On-device AI** — Gemma / LiteRT via `flutter_edge_ai` (Flutter Edge AI)
- **Multi-model** — SmolLM, FastVLM, Gemma 3 1B, Gemma 4 E2B + custom import
- **Voice** — speech-to-text input
- **Screen awareness** — MediaProjection screenshots for vision models
Expand Down Expand Up @@ -91,7 +91,7 @@ There is **no** `android.backup/` — that stale tree was removed.
## Architecture (brief)

1. UI → `ModelOrchestrator.processMessage`
2. RAG + model select + `flutter_gemma` stream
2. RAG + model select + `flutter_edge_ai` stream
3. Tool calls → Dart services or Android `ToolExecutor`
4. Idle / lifecycle unload **after** streaming completes

Expand Down Expand Up @@ -133,6 +133,6 @@ MIT — see [LICENSE](LICENSE).
## Acknowledgments

- [Flutter](https://flutter.dev/)
- [flutter_gemma](https://pub.dev/packages/flutter_gemma)
- [flutter_edge_ai](https://pub.dev/packages/flutter_edge_ai)
- [Gemma](https://ai.google.dev/gemma)
- [HuggingFace](https://huggingface.co/)
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ UI (screens / widgets)
│
ModelOrchestrator ←── MemoryService (RAG), MCP, Tasks, Notes
│
flutter_gemma / LiteRT-LM
flutter_edge_ai / LiteRT-LM
│
Platform channels → Android (tools, screenshot, assistant key)
```
Expand Down
10 changes: 6 additions & 4 deletions lib/main.dart
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import 'dart:async';

import 'package:flutter/material.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_gemma_litertlm/flutter_gemma_litertlm.dart';
import 'package:flutter_gemma_mediapipe/flutter_gemma_mediapipe.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';
import 'package:flutter_edge_ai_mediapipe/flutter_edge_ai_mediapipe.dart';
import 'package:flutter_edge_ai_speech/flutter_edge_ai_speech.dart';
import 'package:nova_assistant/services/model_orchestrator.dart';
import 'package:nova_assistant/services/model_manager.dart';
import 'package:nova_assistant/services/image_generation_service.dart';
Expand Down Expand Up @@ -55,8 +56,9 @@ void main() async {
);
// #endregion

await FlutterGemma.initialize(
await FlutterEdgeAi.initialize(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: TTS initialization failure skips all core service initialization

If FlutterEdgeAi.initialize() throws (e.g., TTS backend registration fails), the entire try block (lines 42-98) fails and the catch at line 96 only logs the error. ModelManager.instance.initialize(), _repairModels(), and all Future.wait service initializations are never executed. The app launches without models, memory, tasks, notes, notifications, TTS, MCP, etc.

Wrap the FlutterEdgeAi.initialize call in its own try-catch so failures don't block the rest of startup, or move TTS backend registration to a separate deferred initialization.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

inferenceEngines: const [LiteRtLmEngine(), MediaPipeEngine()],
ttsBackends: const [LiteRtTtsBackend()],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High · Core services remain uninitialized

When TTS backend registration or initialization throws during startup, FlutterEdgeAi.initialize propagates the error into main's shared startup catch, which skips ModelManager.initialize, _repairModels, and the remaining service initialization; the app launches without its normal model and core services.

Handle TTS initialization failure separately so model and service initialization can continue.

Also at lib/main.dart:62.

Prompt for AI agents
In `lib/main.dart` at line 61:

**Core services remain uninitialized**

When TTS backend registration or initialization throws during startup, `FlutterEdgeAi.initialize` propagates the error into `main`'s shared startup catch, which skips `ModelManager.initialize`, `_repairModels`, and the remaining service initialization; the app launches without its normal model and core services.

Handle TTS initialization failure separately so model and service initialization can continue.

Also at `lib/main.dart:62`.

Fix in Cursor

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: TTS backend registration failure propagates and blocks startup

The ttsBackends: const [LiteRtTtsBackend()] parameter is new in Flutter Edge AI 2.0. If TTS backend initialization fails (missing model, native lib issues, etc.), the exception propagates out of FlutterEdgeAi.initialize() and the shared catch block at line 96 catches it, skipping all subsequent initialization.

Handle TTS initialization separately so model and service initialization can continue even if TTS fails.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

maxDownloadRetries: 3,
);

Expand Down
2 changes: 1 addition & 1 deletion lib/models/external_tool.dart
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import 'dart:convert';
import 'dart:io';

import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';

enum ExternalToolType { http, local, mcp }

Expand Down
2 changes: 1 addition & 1 deletion lib/models/litert_model_catalog.dart
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/models/model_info.dart';
import 'package:nova_assistant/services/huggingface_hub_service.dart';

Expand Down
2 changes: 1 addition & 1 deletion lib/models/model_info.dart
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/models/litert_model_catalog.dart';

enum NovaModel {
Expand Down
2 changes: 1 addition & 1 deletion lib/models/uncensored_model_catalog.dart
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/services/huggingface_hub_service.dart';

/// Curated uncensored chat models that ship LiteRT-native assets
Expand Down
2 changes: 1 addition & 1 deletion lib/screens/assistant_screen.dart
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import 'package:nova_assistant/services/user_preferences_service.dart';
import 'package:nova_assistant/services/mcp_service.dart';
import 'package:nova_assistant/platform/screenshot_service.dart';
import 'package:nova_assistant/platform/overlay_service.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/tools/tool_definitions.dart';
import 'package:nova_assistant/widgets/in_chat_search_bar.dart';
import 'package:nova_assistant/screens/chat_history_screen.dart';
Expand Down
2 changes: 1 addition & 1 deletion lib/screens/custom_model_import_sheet.dart
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import 'package:file_picker/file_picker.dart';
import 'package:flutter/material.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/models/model_info.dart';
import 'package:nova_assistant/services/model_manager.dart';
import 'package:nova_assistant/utils/message_limits.dart';
Expand Down
2 changes: 1 addition & 1 deletion lib/screens/settings_screen.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import 'dart:io';
import 'package:file_picker/file_picker.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:package_info_plus/package_info_plus.dart';
import 'package:path/path.dart' as p;
import 'package:path_provider/path_provider.dart';
Expand Down
2 changes: 1 addition & 1 deletion lib/services/huggingface_hub_service.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import 'dart:convert';
import 'dart:io';

import 'package:flutter/foundation.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/services/model_manager.dart';

/// A Hugging Face Hub model search hit.
Expand Down
2 changes: 1 addition & 1 deletion lib/services/mcp_service.dart
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import 'dart:convert';
import 'dart:io';

import 'package:flutter/foundation.dart';
import 'package:flutter_gemma/flutter_gemma.dart' as gemma;
import 'package:flutter_edge_ai/flutter_edge_ai.dart' as gemma;
import 'package:path_provider/path_provider.dart';
import 'package:shared_preferences/shared_preferences.dart';
import 'package:uuid/uuid.dart';
Expand Down
20 changes: 10 additions & 10 deletions lib/services/model_manager.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import 'dart:io';

import 'package:crypto/crypto.dart';
import 'package:flutter/foundation.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:path_provider/path_provider.dart';
import 'package:shared_preferences/shared_preferences.dart';
import 'package:path/path.dart' as p;
Expand Down Expand Up @@ -815,7 +815,7 @@ class ModelManager {
}

try {
final builder = FlutterGemma.installModel(
final builder = FlutterEdgeAi.installModel(
modelType: modelType,
fileType: fileType,
).fromFile(canonicalPath);
Expand Down Expand Up @@ -1081,12 +1081,12 @@ class ModelManager {
// "Remove" only clears prefs and leaves multi-GB weights on device.
final deleted = await _deleteModelFile(model.fileName);
try {
await FlutterGemma.uninstallModel(modelId);
await FlutterEdgeAi.uninstallModel(modelId);
} catch (e) {
debugPrint('ModelManager: FlutterGemma.uninstallModel: $e');
debugPrint('ModelManager: FlutterEdgeAi.uninstallModel: $e');
}
try {
await FlutterGemma.uninstallModel(model.fileName);
await FlutterEdgeAi.uninstallModel(model.fileName);
} catch (_) {}
_installedModels.removeWhere((m) => m.id == modelId);
_pathCache.remove(model.fileName);
Expand All @@ -1112,9 +1112,9 @@ class ModelManager {

final deleted = await _deleteModelFile(customModel.fileName);
try {
await FlutterGemma.uninstallModel(customModel.fileName);
await FlutterEdgeAi.uninstallModel(customModel.fileName);
} catch (e) {
debugPrint('ModelManager: FlutterGemma.uninstallModel custom: $e');
debugPrint('ModelManager: FlutterEdgeAi.uninstallModel custom: $e');
}
_customModels.removeWhere((m) => m.id == modelId);
_pathCache.remove(customModel.fileName);
Expand Down Expand Up @@ -1238,7 +1238,7 @@ class ModelManager {
/// Register a model already on disk (no download).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Comment still references old package name

The updated comment says registers with flutter_gemma but should say registers with Flutter Edge AI or registers with flutter_edge_ai for consistency with the migration.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

///
/// If [deferInstall] is false (default), registers with flutter_gemma
/// immediately by calling `FlutterGemma.installModel().fromFile().install()`.
/// immediately by calling `FlutterEdgeAi.installModel().fromFile().install()`.
/// This makes the model available for inference.
///
/// If [deferInstall] is true, skips the flutter_gemma registration and just
Expand Down Expand Up @@ -1294,7 +1294,7 @@ class ModelManager {
int registeredSize = actualSize;

try {
await FlutterGemma.installModel(
await FlutterEdgeAi.installModel(
modelType: modelType,
fileType: fileType,
).fromFile(filePath).install();
Expand All @@ -1311,7 +1311,7 @@ class ModelManager {
if (altPath != null && altPath != filePath) {
try {
registeredSize = await File(altPath).length();
await FlutterGemma.installModel(
await FlutterEdgeAi.installModel(
modelType: modelType,
fileType: fileType,
).fromFile(altPath).install();
Expand Down
22 changes: 11 additions & 11 deletions lib/services/model_orchestrator.dart
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import 'dart:io';

import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:path_provider/path_provider.dart';
import 'package:path/path.dart' as p;
import 'package:shared_preferences/shared_preferences.dart';
Expand Down Expand Up @@ -1008,7 +1008,7 @@ class ModelOrchestrator {
data: {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Comment references old package name

The debug log at line 1011 shows 'flutterHasActive': FlutterEdgeAi.hasActiveModel(), but the comment above (line 1008) says flutter_gemma's internal state. Should reference Flutter Edge AI / flutter_edge_ai.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kilocode-bot fix it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't apply this fix automatically this time.

I could not start this fix because the account needs at least $1 in available credits.

Next steps:

  • Add credits to your Kilo account, then retry the fix.
  • Session ID: agent_43a70669-3628-4cde-9a64-126457dd9be3
Technical details
Insufficient credits. Auto-fix requires at least $1 in available credits. Add credits, then retry.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One or more Cloud Agent sessions failed:

  • involvex/nova-assistant (agent_a3ac761f-78cc-4764-a398-0302a61d3cb4): Assistant request was rate limited

'activeModelType': _activeModelType?.name,
'hasActiveModel': _activeModel != null,
'flutterHasActive': FlutterGemma.hasActiveModel(),
'flutterHasActive': FlutterEdgeAi.hasActiveModel(),
},
);
// #endregion
Expand Down Expand Up @@ -1037,7 +1037,7 @@ class ModelOrchestrator {
message: 'Idle release finished',
data: {
'elapsedMs': DateTime.now().millisecondsSinceEpoch - releaseStarted,
'flutterHasActive': FlutterGemma.hasActiveModel(),
'flutterHasActive': FlutterEdgeAi.hasActiveModel(),
},
);
// #endregion
Expand Down Expand Up @@ -1140,7 +1140,7 @@ class ModelOrchestrator {
(modelToLoad != null
? _loadTimeoutFor(modelToLoad)
: const Duration(seconds: 120));
final future = FlutterGemma.getActiveModel(
final future = FlutterEdgeAi.getActiveModel(
maxTokens: tokenLimit,
preferredBackend: preferredBackend,
supportImage: supportImage,
Expand Down Expand Up @@ -1590,9 +1590,9 @@ class ModelOrchestrator {

// Clear flutter_gemma's internal state whenever we need a fresh engine
// (different model OR vision support mismatch).
if (FlutterGemma.hasActiveModel()) {
if (FlutterEdgeAi.hasActiveModel()) {
try {
await FlutterGemma.clearActiveInferenceIdentity();
await FlutterEdgeAi.clearActiveInferenceIdentity();
} catch (e) {
debugPrint('Error clearing active identity: $e');
}
Expand Down Expand Up @@ -2044,9 +2044,9 @@ class ModelOrchestrator {

// Clear BEFORE register — clearing after install() leaves no active model.
await _teardownActiveModel();
if (FlutterGemma.hasActiveModel()) {
if (FlutterEdgeAi.hasActiveModel()) {
try {
await FlutterGemma.clearActiveInferenceIdentity();
await FlutterEdgeAi.clearActiveInferenceIdentity();
} catch (_) {}
}

Expand All @@ -2073,7 +2073,7 @@ class ModelOrchestrator {
'fileName': customModel.fileName,
'backend': backend.name,
'totalMemMb': total,
'hasActive': FlutterGemma.hasActiveModel(),
'hasActive': FlutterEdgeAi.hasActiveModel(),
},
runId: 'post-fix',
);
Expand Down Expand Up @@ -4307,9 +4307,9 @@ class ModelOrchestrator {
}

Future<void> _clearActiveInferenceIdentity() async {
if (!FlutterGemma.hasActiveModel()) return;
if (!FlutterEdgeAi.hasActiveModel()) return;
try {
await FlutterGemma.clearActiveInferenceIdentity();
await FlutterEdgeAi.clearActiveInferenceIdentity();
} catch (e) {
debugPrint('Error clearing active inference identity: $e');
}
Expand Down
2 changes: 1 addition & 1 deletion lib/services/model_update_service.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import 'dart:async';
import 'dart:convert';

import 'package:flutter/foundation.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:http/http.dart' as http;
import 'package:shared_preferences/shared_preferences.dart';
import 'package:nova_assistant/models/litert_model_catalog.dart';
Expand Down
6 changes: 3 additions & 3 deletions lib/services/parallel_session_manager.dart
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import 'dart:async';

import 'package:flutter/foundation.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/models/model_info.dart';
import 'package:nova_assistant/services/model_orchestrator.dart';
import 'package:nova_assistant/services/platform_adaptation_service.dart';
Expand Down Expand Up @@ -290,11 +290,11 @@ class ParallelSessionManager {
/// Get or create a model for inference
Future<InferenceModel?> _getOrCreateModel(NovaModel model) async {
try {
if (!FlutterGemma.hasActiveModel()) {
if (!FlutterEdgeAi.hasActiveModel()) {
return null;
}

return await FlutterGemma.getActiveModel(
return await FlutterEdgeAi.getActiveModel(
maxTokens: _tokenLimitFor(model),
preferredBackend: PreferredBackend.gpu,
supportImage: model.hasVision,
Expand Down
2 changes: 1 addition & 1 deletion lib/services/platform_adaptation_service.dart
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/models/model_info.dart';
import 'package:nova_assistant/services/memory_diagnostics_service.dart';

Expand Down
2 changes: 1 addition & 1 deletion lib/services/session_history_reinjection.dart
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:nova_assistant/models/chat_message.dart';
import 'package:nova_assistant/models/model_info.dart';
import 'package:nova_assistant/utils/message_limits.dart';
Expand Down
Loading
Loading