Add true configuration cache support for git commit hash#2285
Add true configuration cache support for git commit hash#2285Luna712 wants to merge 12 commits intorecloudstream:masterfrom
Conversation
|
@fire-light42 would it be possible to get this reviewed and (hopefully) merged? That way we can also hopefully move forward with #2249 at some point as well. |
|
@fire-light42 sorry for the ping again on this but any chance to review and potentially merge? |
There was a problem hiding this comment.
I very much like your solution with assets!
However it caches the git commits too much. Creating a commit while developing and then rerunning the app will lead to incorrect git commit in app.
I changed some code to include the HEAD and refs/heads files as input files, which will make the job re-run when those files change.
I am no Gradle expert but it seems to work well in my testing. What do you think?
| tasks.register("generateGitHash") { | ||
| val gitHashDir = layout.buildDirectory.dir("generated/git") | ||
| val rootDir = project.rootDir | ||
| outputs.dir(gitHashDir) | ||
|
|
||
| doLast { | ||
| val hash = try { | ||
| // Read the commit hash from .git/HEAD | ||
| val headFile = File(rootDir, ".git/HEAD") | ||
| if (headFile.exists()) { | ||
| val headContent = headFile.readText().trim() | ||
| if (headContent.startsWith("ref:")) { | ||
| val refPath = headContent.substring(5) // e.g., refs/heads/main | ||
| val commitFile = File(rootDir, ".git/$refPath") | ||
| if (commitFile.exists()) commitFile.readText().trim() else "" | ||
| } else headContent // If it's a detached HEAD (commit hash directly) | ||
| } else "" // If .git/HEAD doesn't exist | ||
| } catch (_: Throwable) { | ||
| "" // Just set to an empty string if any exception occurs | ||
| }.take(7) // Get the short commit hash | ||
|
|
||
| val outFile = gitHashDir.get().file("git-hash.txt").asFile | ||
| outFile.parentFile.mkdirs() | ||
| outFile.writeText(hash) | ||
| } | ||
| } | ||
|
|
||
| tasks.withType<MergeSourceSetFolders> { | ||
| if (name.contains("Assets", ignoreCase = true)) { | ||
| dependsOn("generateGitHash") | ||
| val gitHashDir = layout.buildDirectory.dir("generated/git") | ||
|
|
||
| doLast { | ||
| val assetsDir = outputs.files.singleFile | ||
| val gitHashFile = gitHashDir.get().file("git-hash.txt").asFile | ||
| val outFile = File(assetsDir, "git-hash.txt") | ||
| gitHashFile.copyTo(outFile, overwrite = true) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
| tasks.register("generateGitHash") { | |
| val gitHashDir = layout.buildDirectory.dir("generated/git") | |
| val rootDir = project.rootDir | |
| outputs.dir(gitHashDir) | |
| doLast { | |
| val hash = try { | |
| // Read the commit hash from .git/HEAD | |
| val headFile = File(rootDir, ".git/HEAD") | |
| if (headFile.exists()) { | |
| val headContent = headFile.readText().trim() | |
| if (headContent.startsWith("ref:")) { | |
| val refPath = headContent.substring(5) // e.g., refs/heads/main | |
| val commitFile = File(rootDir, ".git/$refPath") | |
| if (commitFile.exists()) commitFile.readText().trim() else "" | |
| } else headContent // If it's a detached HEAD (commit hash directly) | |
| } else "" // If .git/HEAD doesn't exist | |
| } catch (_: Throwable) { | |
| "" // Just set to an empty string if any exception occurs | |
| }.take(7) // Get the short commit hash | |
| val outFile = gitHashDir.get().file("git-hash.txt").asFile | |
| outFile.parentFile.mkdirs() | |
| outFile.writeText(hash) | |
| } | |
| } | |
| tasks.withType<MergeSourceSetFolders> { | |
| if (name.contains("Assets", ignoreCase = true)) { | |
| dependsOn("generateGitHash") | |
| val gitHashDir = layout.buildDirectory.dir("generated/git") | |
| doLast { | |
| val assetsDir = outputs.files.singleFile | |
| val gitHashFile = gitHashDir.get().file("git-hash.txt").asFile | |
| val outFile = File(assetsDir, "git-hash.txt") | |
| gitHashFile.copyTo(outFile, overwrite = true) | |
| } | |
| } | |
| } | |
| abstract class GenerateGitHashTask : DefaultTask() { | |
| @get:InputFile | |
| @get:PathSensitive(PathSensitivity.RELATIVE) | |
| abstract val headFile: RegularFileProperty | |
| @get:InputDirectory | |
| @get:PathSensitive(PathSensitivity.RELATIVE) | |
| abstract val headsDir: DirectoryProperty | |
| @get:OutputDirectory | |
| abstract val outputDir: DirectoryProperty | |
| @TaskAction | |
| fun generate() { | |
| val head = headFile.get().asFile | |
| val hash = try { | |
| if (head.exists()) { | |
| // Read the commit hash from .git/HEAD | |
| val headContent = head.readText().trim() | |
| if (headContent.startsWith("ref:")) { | |
| val refPath = headContent.substring(5) // e.g., refs/heads/main | |
| val commitFile = File(head.parentFile, refPath) | |
| if (commitFile.exists()) commitFile.readText().trim() else "" | |
| } else headContent // If it's a detached HEAD (commit hash directly) | |
| } else "" // If .git/HEAD doesn't exist | |
| } catch (_: Throwable) { | |
| "" // Just set to an empty string if any exception occurs | |
| }.take(7) // Get the short commit hash | |
| val outFile = outputDir.file("git-hash.txt").get().asFile | |
| outFile.parentFile.mkdirs() | |
| outFile.writeText(hash) | |
| } | |
| } | |
| val generateGitHash = tasks.register<GenerateGitHashTask>("generateGitHash") { | |
| val gitDir = layout.projectDirectory.dir("../.git") | |
| headFile.set(gitDir.file("HEAD")) | |
| headsDir.set(gitDir.dir("refs/heads")) | |
| outputDir.set(layout.buildDirectory.dir("generated/git")) | |
| } |
There was a problem hiding this comment.
This seems like a great solution, I will do some testing with it myself in the next day or two to ensure it works fully with configuration cache without issues. But your solution seems like it will to me.
| viewBinding { | ||
| enable = true | ||
| } | ||
|
|
There was a problem hiding this comment.
| androidComponents { | |
| onVariants { variant -> | |
| variant.sources.assets?.addGeneratedSourceDirectory( | |
| generateGitHash, | |
| GenerateGitHashTask::outputDir | |
| ) | |
| } | |
| } |
This prevents the configuration cache from rebuilding every single time there is a new commit, which can drastically improve build performance, especially when there are no Kotlin files changed. I tested this on pre release builds, once with no files changed that is in the build (just changing GitHub workflows) and pre release built in 38 seconds. The second time, I tried just changing an XML resource and it built in about 2 minutes. Third time changing a Kotlin file (which requires recompiling it all) and it ran in about 6 minutes, which is still drastically faster than without configuration cache at all.
There were other ways to do this, I tried numerous ways including:
Using an asset only requires merging new assets every single time which only takes about 3 seconds which is very performant and allows other gradle features aimed at speeding up build to work and allow ensuring that files that don't need rebuilding aren't. This prevents the reclaculating of configuration cache every new commit allowing it to work properly. Runtime performance seems to not be affected at all when reading the asset rather than a
resValue. Seems to have no affect at runtime.This needs to be combined with #2249 to have much impact here on the GitHub builds though.