Adam
df197463eb
remove unused SuppressProcessor.java
2022-05-02 23:34:59 +02:00
Adam
ee3c7956cd
events-server: fix tile grouping, chunking, saving
...
- debounce chunk and tile updates
- make the tile output-dir configurable
- basic test for manual checking
2022-05-01 15:50:44 +02:00
Adam
fb265e39a6
event-server: try sharing a job, so server shuts down at the same time
2022-05-01 15:45:23 +02:00
Adam
e5bfc29697
simplify DebounceProcessor - use for-loop and schedule instead of coroutines
2022-05-01 09:46:53 +02:00
Adam
a78ffb794a
implement versions platform, tidy up and bump some dependencies
2022-05-01 09:42:14 +02:00
Adam
b4ba87e8e0
try fixing tstl task caching
2022-04-24 19:00:37 +02:00
Adam
e05db4398a
re-implement tile saver, without Coroutines or debouncing. It seems to work well.
2022-04-24 19:00:10 +02:00
Adam
041d570aed
re-implement debounce processor... and then realise it's difficult to integrate with Kafka Streams DSL so don't use it
2022-04-24 18:55:18 +02:00
Adam
4b5fb7a944
kafka-streams: experiment with exposing jmx
2022-04-24 11:33:18 +02:00
Adam
12df3b80be
visual test for map chunk grouper
2022-04-24 11:32:56 +02:00
Adam
ee62f9646e
Kafka Streams working again?
2022-04-24 11:32:15 +02:00
Adam
0d81650400
save topology description to disk
2022-04-23 21:56:43 +02:00
Adam
fea0d5bb97
one more group map chunks test
2022-04-23 17:59:52 +02:00
Adam
b81e62e2fd
enable gradle build cache, and improve tasks
2022-04-23 17:59:37 +02:00
Adam
f339333f76
change Kafka admin id warehouse->kafkatorio
2022-04-20 23:17:50 +02:00
Adam
2a3b0de953
more tests for grouping map chunks
2022-04-20 22:58:06 +02:00
Adam
2287582880
tidy up code
2022-04-20 17:54:54 +02:00
Adam
b0a6151c36
move from schema2 to schema
2022-04-20 17:18:09 +02:00
Adam
6bfcd30558
use tuples for colours
2022-04-20 17:14:00 +02:00
Adam
1c55fae236
tidying up schema files
2022-04-20 17:03:23 +02:00
Adam
d1353eec60
fix broken tests
2022-04-20 12:58:04 +02:00
Adam
8bf393f9c3
start tidying up after schema->schema2 migration
2022-04-20 11:28:30 +02:00
Adam
3fff57d46d
biggest refactor ever...
...
- use kxs-ts-gen for TypeScript schemas
- update schema files, so they're tidier (no need for typescript-generator workarounds)
- version bumps (kotlin, gradle, tstl, factorio, kotest)
- improve kxs binary format (use okio)
- move schema files package ('events' wasn't needed)
- tidy up topic creation, naming. base it off classes, make util functions
2022-04-20 10:17:41 +02:00
Adam
9e55a33882
disable max.task.idle.ms setting (I don't know what it's supposed to do, and setting it high causes OOM)
2022-03-14 22:21:17 +01:00
Adam
f1d15085e6
reduce log output when saving tiles
2022-03-14 22:20:39 +01:00
Adam
057eee6a68
- remove coroutines (most packets will only have one group, so it doesn't help much)
...
- don't materialize to named stores (unneeded)
2022-03-14 00:00:04 +01:00
Adam
0ebf492b42
max task idle ms - try fixing ktable-ktable join...
2022-03-13 23:58:41 +01:00
Adam
6bb514afc4
still a bit janky (the prototype/tiles join ordering isn't right) but it mostly works, and is performant
2022-03-13 12:37:43 +01:00
Adam
8bb73b8291
remove commented out code, try using materializedAs to fix KTable-KTable join not working, make 'saveMapTiles' more encapsulated
2022-03-13 11:06:30 +01:00
Adam
3a10485ed4
bump some versions, pin some npm dependencies to satisfy GitHub depend-a-bot, remove protobuf (unused)
2022-03-12 22:00:58 +01:00
Adam
b8ba8b7c9e
- tidy up groupMapChunks.kt
...
- add ticks to all Kafkatorio Packets
- add Kafka Streams test for groupMapChunks.kt
2022-03-12 16:42:24 +01:00
Adam
bfdee32d61
bump kotka-streams versions (fix deserialization bug, because of nullable val)
2022-03-12 02:10:40 +01:00
Adam
19bfe3a232
shutdown all KafkaStream apps if one crashes
2022-03-12 01:39:19 +01:00
Adam
7a46ea7b75
KxsBinaryTests
2022-03-12 01:38:13 +01:00
Adam
b6f8d2e422
KafkaAdmin - make all event-update topics
2022-03-12 01:38:00 +01:00
Adam
e3954c184a
code tidy up
2022-03-12 01:37:35 +01:00
Adam
e630ce3702
Gradle version bump, and fiddling with some gradle config. Add a 'hardReset' Gradle task. Add a 'docker remove' task.
2022-03-09 21:41:00 +01:00
Adam
a9782a4d90
version bumps, schema tweaks, code tweaks,
2022-03-04 23:54:45 +01:00
Adam
b055397d25
- wicked sick size optimisation of map-tiles update packet
...
- fix ts-generator recursive value class unwrapping
- move kafka state dir
- try fixing kafka streams for tile generation (simplifying it, and fixing some nullability problems)
2022-03-02 18:39:03 +01:00
Adam
ebb8482201
make factorio ID value classes, nicer, and remove 'EncodeDefault' from colours
2022-02-27 01:02:33 +01:00
Adam
99a7167485
improve kafka consumer group gradle task
2022-02-27 01:01:24 +01:00
Adam
b42eaaa67e
create KafkaConsumers reset task
2022-02-26 17:34:31 +01:00
Adam
066f68ff4d
event debouncing / throttling basically working now
2022-02-24 21:17:51 +01:00
Adam
494913d3b9
fix init of global vars, make MapTile DTO more concise, and version bump
2022-02-21 15:36:12 +01:00
Adam
7481c8162a
fixing 'when' statements, tidying up some code,
2022-02-20 22:30:56 +01:00
Adam
dd45bece53
model updates, and fix tests
2022-02-20 22:16:21 +01:00
Adam
c53d1d7c87
refactoring (I can't remember what... it's untested for sure), and using the latest things from event-schema
2022-02-20 12:36:53 +01:00
Adam
a8337552e6
use compact size for collections (I think I tried this already and it failed - but I'm going to wipe all data this time, so hopefully it works)
2022-02-20 12:32:17 +01:00
Adam
2aed3a670c
kxsBinary.kt - formatting
2022-02-20 12:31:39 +01:00
Adam
254c600309
use static enum values, not values() (better performance)
2022-02-20 12:30:08 +01:00
