Adam
0d81650400
save topology description to disk
2022-04-23 21:56:43 +02:00
Adam
fea0d5bb97
one more group map chunks test
2022-04-23 17:59:52 +02:00
Adam
b81e62e2fd
enable gradle build cache, and improve tasks
2022-04-23 17:59:37 +02:00
Adam
f339333f76
change Kafka admin id warehouse->kafkatorio
2022-04-20 23:17:50 +02:00
Adam
2a3b0de953
more tests for grouping map chunks
2022-04-20 22:58:06 +02:00
Adam
2287582880
tidy up code
2022-04-20 17:54:54 +02:00
Adam
b0a6151c36
move from schema2 to schema
2022-04-20 17:18:09 +02:00
Adam
6bfcd30558
use tuples for colours
2022-04-20 17:14:00 +02:00
Adam
1c55fae236
tidying up schema files
2022-04-20 17:03:23 +02:00
Adam
d1353eec60
fix broken tests
2022-04-20 12:58:04 +02:00
Adam
8bf393f9c3
start tidying up after schema->schema2 migration
2022-04-20 11:28:30 +02:00
Adam
3fff57d46d
biggest refactor ever...
...
- use kxs-ts-gen for TypeScript schemas
- update schema files, so they're tidier (no need for typescript-generator workarounds)
- version bumps (kotlin, gradle, tstl, factorio, kotest)
- improve kxs binary format (use okio)
- move schema files package ('events' wasn't needed)
- tidy up topic creation, naming. base it off classes, make util functions
2022-04-20 10:17:41 +02:00
Adam
9e55a33882
disable max.task.idle.ms setting (I don't know what it's supposed to do, and setting it high causes OOM)
2022-03-14 22:21:17 +01:00
Adam
f1d15085e6
reduce log output when saving tiles
2022-03-14 22:20:39 +01:00
Adam
057eee6a68
- remove coroutines (most packets will only have one group, so it doesn't help much)
...
- don't materialize to named stores (unneeded)
2022-03-14 00:00:04 +01:00
Adam
0ebf492b42
max task idle ms - try fixing ktable-ktable join...
2022-03-13 23:58:41 +01:00
Adam
6bb514afc4
still a bit janky (the prototype/tiles join ordering isn't right) but it mostly works, and is performant
2022-03-13 12:37:43 +01:00
Adam
8bb73b8291
remove commented out code, try using materializedAs to fix KTable-KTable join not working, make 'saveMapTiles' more encapsulated
2022-03-13 11:06:30 +01:00
Adam
3a10485ed4
bump some versions, pin some npm dependencies to satisfy GitHub depend-a-bot, remove protobuf (unused)
2022-03-12 22:00:58 +01:00
Adam
b8ba8b7c9e
- tidy up groupMapChunks.kt
...
- add ticks to all Kafkatorio Packets
- add Kafka Streams test for groupMapChunks.kt
2022-03-12 16:42:24 +01:00
Adam
19bfe3a232
shutdown all KafkaStream apps if one crashes
2022-03-12 01:39:19 +01:00
Adam
7a46ea7b75
KxsBinaryTests
2022-03-12 01:38:13 +01:00
Adam
b6f8d2e422
KafkaAdmin - make all event-update topics
2022-03-12 01:38:00 +01:00
Adam
e3954c184a
code tidy up
2022-03-12 01:37:35 +01:00
Adam
e630ce3702
Gradle version bump, and fiddling with some gradle config. Add a 'hardReset' Gradle task. Add a 'docker remove' task.
2022-03-09 21:41:00 +01:00
Adam
a9782a4d90
version bumps, schema tweaks, code tweaks,
2022-03-04 23:54:45 +01:00
Adam
b055397d25
- wicked sick size optimisation of map-tiles update packet
...
- fix ts-generator recursive value class unwrapping
- move kafka state dir
- try fixing kafka streams for tile generation (simplifying it, and fixing some nullability problems)
2022-03-02 18:39:03 +01:00
Adam
ebb8482201
make factorio ID value classes, nicer, and remove 'EncodeDefault' from colours
2022-02-27 01:02:33 +01:00
Adam
066f68ff4d
event debouncing / throttling basically working now
2022-02-24 21:17:51 +01:00
Adam
494913d3b9
fix init of global vars, make MapTile DTO more concise, and version bump
2022-02-21 15:36:12 +01:00
Adam
7481c8162a
fixing 'when' statements, tidying up some code,
2022-02-20 22:30:56 +01:00
Adam
dd45bece53
model updates, and fix tests
2022-02-20 22:16:21 +01:00
Adam
c53d1d7c87
refactoring (I can't remember what... it's untested for sure), and using the latest things from event-schema
2022-02-20 12:36:53 +01:00
Adam
a8337552e6
use compact size for collections (I think I tried this already and it failed - but I'm going to wipe all data this time, so hopefully it works)
2022-02-20 12:32:17 +01:00
Adam
2aed3a670c
kxsBinary.kt - formatting
2022-02-20 12:31:39 +01:00
Adam
254c600309
use static enum values, not values() (better performance)
2022-02-20 12:30:08 +01:00
Adam
d3c7abff95
rm unused class (moved to kotka-streams)
2022-02-20 12:21:46 +01:00
Adam
6d6429ebac
rename&move KafkatorioKafkaAdmin.kt
2022-02-13 12:01:12 +01:00
Adam
7141d7e5f7
- try making a factorio mod portal api client, for auto releasing
...
- make webmapTileDir a configurable variable
2022-02-13 10:42:13 +01:00
Adam
ec1f1bc200
reduce logging for saving tiles
2022-02-09 23:44:21 +01:00
Adam
b37da373ae
split up the tile-chunk-grouping and save-tile applications
2022-02-09 23:38:12 +01:00
Adam
66fa1fb53a
reuse the packet-splitter topics for the tile-saver job
2022-02-09 23:02:00 +01:00
Adam
ba03f761e0
zoom zoom faster - make 3 different Kafka consumers for player/tile/packet updates
2022-02-08 17:56:34 +01:00
Adam
215ef66fbd
delete old code, try and tidy up coroutines launching
2022-02-08 14:55:11 +01:00
Adam
1af5912477
even more working - generate zoomed tiles (but also it doesn't work - it stops making new tiles at some point).
2022-02-08 14:10:13 +01:00
Adam
5a59b3bd6d
workinggggg!!!!!!!
...
- fix map tiles proto colours KTable
- test and tidy up Colour/ColourHex
- reset event-server application
2022-02-08 11:51:16 +01:00
Adam
bdb2d9577f
so close to working performantly.... just the tiles are transparent???
...
- bump some versions
- MapTile no longer has objectName field
- fix nulls/optional fields in FactorioConfigurationUpdate
- use bytes to encode rgba colours - waaaayyyy smaller
- increase Kafka record size to ~10mb
2022-02-08 01:35:52 +01:00
Adam
ea0bfb350b
- MapTile/LuaTile no longer a FactorioObjectData (it doesn't get sent as a packet)
...
- trying to optimise the topology so it's faster... almost there but the records are now too large
- map tile pngs, multiple zoom levels
- use coroutines for faster map tiles
2022-02-06 21:05:37 +01:00
Adam
d75233e71e
experimenting with a custom debouncer
2022-02-05 12:21:11 +01:00
Adam
f108431033
- bump kafka sdk version
...
- fix kafka admin so it doesn't try making topics always
- move some stuff to Kotka Streams
- trying to optimise and debounce with windows... but it didn't work
2022-02-05 12:20:25 +01:00
