GitBucket
+ @defining(AutoUpdate.getCurrentVersion){ version =>
+ @version.majorVersion.@version.minorVersion
+ }
+ change code !!!!!!!!!!!!!!!!
+ GitBucket
+ @defining(AutoUpdate.getCurrentVersion){ version =>
+ @version.majorVersion.@version.minorVersion
+ }
+ change code !!!!!!!!!!!!!!!!
+
+:
+```
+
+If JRebel is doing is correctly installed you will see a notice for you:
+
+```
+1. Waiting for source changes... (press enter to interrupt)
+2016-01-03 21:48:42 JRebel: Reloading class 'gitbucket.core.html.main$'.
+[info] Wrote rebel.xml to /git/gitbucket/target/scala-2.11/resource_managed/main/rebel.xml
+[info] Compiling 1 Scala source to /git/gitbucket/target/scala-2.11/classes...
+[success] Total time: 3 s, completed Jan 3, 2016 9:48:55 PM
+2. Waiting for source changes... (press enter to interrupt)
+```
+
+And you reload browser, JRebel give notice of that it has reloaded classes:
+
+```
+[success] Total time: 3 s, completed Jan 3, 2016 9:48:55 PM
+2. Waiting for source changes... (press enter to interrupt)
+2016-01-03 21:49:13 JRebel: Reloading class 'gitbucket.core.html.main$'.
+```
+
+## 6. Limitations
+
+JRebel is nearly always able to eliminate the need to explicitly reload your container after a code change. However, if you change any of your routes patterns, there is nothing JRebel can do, you will have to run `jetty:start`.
diff --git a/doc/readme.md b/doc/readme.md
index bf69b5c..f8b23de 100644
--- a/doc/readme.md
+++ b/doc/readme.md
@@ -3,9 +3,10 @@
* [How to run from source tree](how_to_run.md)
* [Directory Structure](directory.md)
* [Mapping and Validation](validation.md)
- * Authentication in Controller (not yet)
+ * [Authentication in Controller](authenticator.md)
* [About Action in Issue Comment](comment_action.md)
* [Activity Types](activity.md)
* [Notification Email](notification.md)
* [Automatic Schema Updating](auto_update.md)
* [Release Operation](release.md)
+ * [JRebel integration (optional)](jrebel.md)
diff --git a/doc/release.md b/doc/release.md
index 186f251..682dc88 100644
--- a/doc/release.md
+++ b/doc/release.md
@@ -6,24 +6,29 @@
Note to update version number in files below:
-### src/main/scala/gitbucket/core/servlet/AutoUpdate.scala
+### build.sbt
```scala
-object AutoUpdate {
-
- /**
- * The history of versions. A head of this sequence is the current BitBucket version.
- */
- val versions = Seq(
- new Version(3, 3), // <---- add this line!!
- new Version(3, 2),
+val Organization = "gitbucket"
+val Name = "gitbucket"
+val GitBucketVersion = "4.0.0" // <---- update version!!
+val ScalatraVersion = "2.4.0"
+val JettyVersion = "9.3.6.v20151106"
```
-### env.sh
+### src/main/scala/gitbucket/core/GitBucketCoreModule.scala
-```bash
-#!/bin/sh
-export GITBUCKET_VERSION=3.3.0 # <---- update here!!
+```scala
+object GitBucketCoreModule extends Module("gitbucket-core",
+ new Version("4.0.0",
+ new LiquibaseMigration("update/gitbucket-core_4.0.xml"),
+ new SqlMigration("update/gitbucket-core_4.0.sql")
+ ),
+ // add new version definition
+ new Version("4.1.0",
+ new LiquibaseMigration("update/gitbucket-core_4.1.xml")
+ )
+)
```
Generate release files
@@ -33,18 +38,18 @@
### Make release war file
-Run `release/make-release-war.sh`. The release war file is generated into `target/scala-2.11/gitbucket.war`.
+Run `sbt executable`. The release war file and fingerprint are generated into `target/executable/gitbucket.war`.
```bash
-$ cd release
-$ ./make-release-war.sh
+$ sbt executable
```
### Deploy assembly jar file
-For plug-in development, we have to publish the assembly jar file to the public Maven repository by `release/deploy-assembly-jar.sh`.
+For plug-in development, we have to publish the GitBucket jar file to the Maven central repository as well. At first, hit following command to publish artifacts to the sonatype OSS repository:
```bash
-$ cd release/
-$ ./deploy-assembly-jar.sh
+$ sbt publish-signed
```
+
+Then operate release sequence at https://oss.sonatype.org/.
diff --git a/embed-jetty/javax.servlet-3.0.0.v201112011016.jar b/embed-jetty/javax.servlet-3.0.0.v201112011016.jar
deleted file mode 100644
index b135409..0000000
--- a/embed-jetty/javax.servlet-3.0.0.v201112011016.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-continuation-8.1.16.v20140903.jar b/embed-jetty/jetty-continuation-8.1.16.v20140903.jar
deleted file mode 100644
index ce1acb1..0000000
--- a/embed-jetty/jetty-continuation-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-http-8.1.16.v20140903.jar b/embed-jetty/jetty-http-8.1.16.v20140903.jar
deleted file mode 100644
index 30189c7..0000000
--- a/embed-jetty/jetty-http-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-io-8.1.16.v20140903.jar b/embed-jetty/jetty-io-8.1.16.v20140903.jar
deleted file mode 100644
index a9afd7c..0000000
--- a/embed-jetty/jetty-io-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-security-8.1.16.v20140903.jar b/embed-jetty/jetty-security-8.1.16.v20140903.jar
deleted file mode 100644
index e5bde43..0000000
--- a/embed-jetty/jetty-security-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-server-8.1.16.v20140903.jar b/embed-jetty/jetty-server-8.1.16.v20140903.jar
deleted file mode 100644
index ae8ac55..0000000
--- a/embed-jetty/jetty-server-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-servlet-8.1.16.v20140903.jar b/embed-jetty/jetty-servlet-8.1.16.v20140903.jar
deleted file mode 100644
index eb2fa57..0000000
--- a/embed-jetty/jetty-servlet-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-util-8.1.16.v20140903.jar b/embed-jetty/jetty-util-8.1.16.v20140903.jar
deleted file mode 100644
index 5c3c346..0000000
--- a/embed-jetty/jetty-util-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-webapp-8.1.16.v20140903.jar b/embed-jetty/jetty-webapp-8.1.16.v20140903.jar
deleted file mode 100644
index 85fd7e0..0000000
--- a/embed-jetty/jetty-webapp-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/jetty-xml-8.1.16.v20140903.jar b/embed-jetty/jetty-xml-8.1.16.v20140903.jar
deleted file mode 100644
index 1e485de..0000000
--- a/embed-jetty/jetty-xml-8.1.16.v20140903.jar
+++ /dev/null
Binary files differ
diff --git a/embed-jetty/update.sh b/embed-jetty/update.sh
deleted file mode 100755
index 7d1cfd7..0000000
--- a/embed-jetty/update.sh
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/bin/bash
-version=$1
-output_dir=`dirname $0`
-git rm -f ${output_dir}/jetty-*.jar
-for name in 'io' 'servlet' 'xml' 'continuation' 'security' 'util' 'http' 'server' 'webapp'
-do
- jar_filename="jetty-${name}-${version}.jar"
- wget "http://repo1.maven.org/maven2/org/eclipse/jetty/jetty-${name}/${version}/${jar_filename}" -O ${output_dir}/${jar_filename}
-done
-git add ${output_dir}/*.jar
-git commit
diff --git a/env.sh b/env.sh
deleted file mode 100644
index d6cd233..0000000
--- a/env.sh
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/sh
-export GITBUCKET_VERSION=3.5.0-SNAPSHOT
diff --git a/gitbucket-assembly.iml b/gitbucket-assembly.iml
deleted file mode 100644
index 3f0a572..0000000
--- a/gitbucket-assembly.iml
+++ /dev/null
@@ -1,14 +0,0 @@
-
-", + play.twirl.api.HtmlFormat.escape(new String(c)).body, + "", "
/:user/:repo.git/
+ */
+ private val githubRepositoryPattern = """^/[^/]+/[^/]+\.git/.*""".r
+
+ override def init(filterConfig: FilterConfig) = {}
+
+ override def doFilter(req: ServletRequest, res: ServletResponse, chain: FilterChain) = {
+ implicit val request = req.asInstanceOf[HttpServletRequest]
+ val response = res.asInstanceOf[HttpServletResponse]
+ val requestPath = request.getRequestURI.substring(request.getContextPath.length)
+ requestPath match {
+ case githubRepositoryPattern() =>
+ response.sendRedirect(baseUrl + "/git" + requestPath)
+
+ case _ =>
+ chain.doFilter(req, res)
+ }
+ }
+
+ override def destroy() = {}
+
+}
diff --git a/src/main/scala/gitbucket/core/servlet/GitAuthenticationFilter.scala b/src/main/scala/gitbucket/core/servlet/GitAuthenticationFilter.scala
new file mode 100644
index 0000000..59cd058
--- /dev/null
+++ b/src/main/scala/gitbucket/core/servlet/GitAuthenticationFilter.scala
@@ -0,0 +1,116 @@
+package gitbucket.core.servlet
+
+import javax.servlet._
+import javax.servlet.http._
+import gitbucket.core.plugin.{GitRepositoryFilter, GitRepositoryRouting, PluginRegistry}
+import gitbucket.core.service.SystemSettingsService.SystemSettings
+import gitbucket.core.service.{RepositoryService, AccountService, SystemSettingsService}
+import gitbucket.core.util.{Keys, Implicits, AuthUtil}
+import org.slf4j.LoggerFactory
+import Implicits._
+
+/**
+ * Provides BASIC Authentication for [[GitRepositoryServlet]].
+ */
+class GitAuthenticationFilter extends Filter with RepositoryService with AccountService with SystemSettingsService {
+
+ private val logger = LoggerFactory.getLogger(classOf[GitAuthenticationFilter])
+
+ def init(config: FilterConfig) = {}
+
+ def destroy(): Unit = {}
+
+ def doFilter(req: ServletRequest, res: ServletResponse, chain: FilterChain): Unit = {
+ val request = req.asInstanceOf[HttpServletRequest]
+ val response = res.asInstanceOf[HttpServletResponse]
+
+ val wrappedResponse = new HttpServletResponseWrapper(response){
+ override def setCharacterEncoding(encoding: String) = {}
+ }
+
+ val isUpdating = request.getRequestURI.endsWith("/git-receive-pack") || "service=git-receive-pack".equals(request.getQueryString)
+ val settings = loadSystemSettings()
+
+ try {
+ PluginRegistry().getRepositoryRouting(request.gitRepositoryPath).map { case GitRepositoryRouting(_, _, filter) =>
+ // served by plug-ins
+ pluginRepository(request, wrappedResponse, chain, settings, isUpdating, filter)
+
+ }.getOrElse {
+ // default repositories
+ defaultRepository(request, wrappedResponse, chain, settings, isUpdating)
+ }
+ } catch {
+ case ex: Exception => {
+ logger.error("error", ex)
+ AuthUtil.requireAuth(response)
+ }
+ }
+ }
+
+ private def pluginRepository(request: HttpServletRequest, response: HttpServletResponse, chain: FilterChain,
+ settings: SystemSettings, isUpdating: Boolean, filter: GitRepositoryFilter): Unit = {
+ implicit val r = request
+
+ val account = for {
+ auth <- Option(request.getHeader("Authorization"))
+ Array(username, password) = AuthUtil.decodeAuthHeader(auth).split(":", 2)
+ account <- authenticate(settings, username, password)
+ } yield {
+ request.setAttribute(Keys.Request.UserName, account.userName)
+ account
+ }
+
+ if(filter.filter(request.gitRepositoryPath, account.map(_.userName), settings, isUpdating)){
+ chain.doFilter(request, response)
+ } else {
+ AuthUtil.requireAuth(response)
+ }
+ }
+
+ private def defaultRepository(request: HttpServletRequest, response: HttpServletResponse, chain: FilterChain,
+ settings: SystemSettings, isUpdating: Boolean): Unit = {
+ val action = request.paths match {
+ case Array(_, repositoryOwner, repositoryName, _*) =>
+ Database() withSession { implicit session =>
+ getRepository(repositoryOwner, repositoryName.replaceFirst("\\.wiki\\.git$|\\.git$", "")) match {
+ case Some(repository) => {
+ val execute = if (!isUpdating && !repository.repository.isPrivate && settings.allowAnonymousAccess) {
+ // Authentication is not required
+ true
+ } else {
+ // Authentication is required
+ val passed = for {
+ auth <- Option(request.getHeader("Authorization"))
+ Array(username, password) = AuthUtil.decodeAuthHeader(auth).split(":", 2)
+ account <- authenticate(settings, username, password)
+ } yield if (isUpdating || repository.repository.isPrivate) {
+ if (hasDeveloperRole(repository.owner, repository.name, Some(account))) {
+ request.setAttribute(Keys.Request.UserName, account.userName)
+ true
+ } else false
+ } else true
+ passed.getOrElse(false)
+ }
+
+ if (execute) {
+ () => chain.doFilter(request, response)
+ } else {
+ () => AuthUtil.requireAuth(response)
+ }
+ }
+ case None => () => {
+ logger.debug(s"Repository ${repositoryOwner}/${repositoryName} is not found.")
+ response.sendError(HttpServletResponse.SC_NOT_FOUND)
+ }
+ }
+ }
+ case _ => () => {
+ logger.debug(s"Not enough path arguments: ${request.paths}")
+ response.sendError(HttpServletResponse.SC_NOT_FOUND)
+ }
+ }
+
+ action()
+ }
+}
\ No newline at end of file
diff --git a/src/main/scala/gitbucket/core/servlet/GitLfsTransferServlet.scala b/src/main/scala/gitbucket/core/servlet/GitLfsTransferServlet.scala
new file mode 100644
index 0000000..4c31884
--- /dev/null
+++ b/src/main/scala/gitbucket/core/servlet/GitLfsTransferServlet.scala
@@ -0,0 +1,83 @@
+package gitbucket.core.servlet
+
+import java.io.{File, FileInputStream, FileOutputStream}
+import java.text.MessageFormat
+import javax.servlet.http.{HttpServlet, HttpServletRequest, HttpServletResponse}
+
+import gitbucket.core.util.{FileUtil, StringUtil}
+import org.apache.commons.io.{FileUtils, IOUtils}
+import org.json4s.jackson.Serialization._
+import org.apache.http.HttpStatus
+import gitbucket.core.util.ControlUtil._
+
+/**
+ * Provides GitLFS Transfer API
+ * https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md
+ */
+class GitLfsTransferServlet extends HttpServlet {
+
+ private implicit val jsonFormats = gitbucket.core.api.JsonFormat.jsonFormats
+ private val LongObjectIdLength = 32
+ private val LongObjectIdStringLength = LongObjectIdLength * 2
+
+ override protected def doGet(req: HttpServletRequest, res: HttpServletResponse): Unit = {
+ for {
+ (owner, repository, oid) <- getPathInfo(req, res) if checkToken(req, oid)
+ } yield {
+ val file = new File(FileUtil.getLfsFilePath(owner, repository, oid))
+ if(file.exists()){
+ res.setStatus(HttpStatus.SC_OK)
+ res.setContentType("application/octet-stream")
+ res.setContentLength(file.length.toInt)
+ using(new FileInputStream(file), res.getOutputStream){ (in, out) =>
+ IOUtils.copy(in, out)
+ out.flush()
+ }
+ } else {
+ sendError(res, HttpStatus.SC_NOT_FOUND,
+ MessageFormat.format("Object ''{0}'' not found", oid))
+ }
+ }
+ }
+
+ override protected def doPut(req: HttpServletRequest, res: HttpServletResponse): Unit = {
+ for {
+ (owner, repository, oid) <- getPathInfo(req, res) if checkToken(req, oid)
+ } yield {
+ val file = new File(FileUtil.getLfsFilePath(owner, repository, oid))
+ FileUtils.forceMkdir(file.getParentFile)
+ using(req.getInputStream, new FileOutputStream(file)){ (in, out) =>
+ IOUtils.copy(in, out)
+ }
+ res.setStatus(HttpStatus.SC_OK)
+ }
+ }
+
+ private def checkToken(req: HttpServletRequest, oid: String): Boolean = {
+ val token = req.getHeader("Authorization")
+ if(token != null){
+ val Array(expireAt, targetOid) = StringUtil.decodeBlowfish(token).split(" ")
+ oid == targetOid && expireAt.toLong > System.currentTimeMillis
+ } else {
+ false
+ }
+ }
+
+ private def getPathInfo(req: HttpServletRequest, res: HttpServletResponse): Option[(String, String, String)] = {
+ req.getRequestURI.substring(1).split("/") match {
+ case Array(_, owner, repository, oid) => Some((owner, repository, oid))
+ case _ => None
+ }
+ }
+
+ private def sendError(res: HttpServletResponse, status: Int, message: String): Unit = {
+ res.setStatus(status)
+ using(res.getWriter()){ out =>
+ out.write(write(GitLfs.Error(message)))
+ out.flush()
+ }
+ }
+
+}
+
+
diff --git a/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala b/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala
index 88b8124..5ca72be 100644
--- a/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala
+++ b/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala
@@ -1,54 +1,45 @@
package gitbucket.core.servlet
+import java.io.File
+import java.util.Date
+
import gitbucket.core.api
-import gitbucket.core.model.Session
+import gitbucket.core.model.WebHook
+import gitbucket.core.plugin.{GitRepositoryRouting, PluginRegistry}
import gitbucket.core.service.IssuesService.IssueSearchCondition
import gitbucket.core.service.WebHookService._
import gitbucket.core.service._
import gitbucket.core.util.ControlUtil._
import gitbucket.core.util.Implicits._
-import gitbucket.core.util.JGitUtil.CommitInfo
import gitbucket.core.util._
-
import org.eclipse.jgit.api.Git
import org.eclipse.jgit.http.server.GitServlet
import org.eclipse.jgit.lib._
import org.eclipse.jgit.transport._
import org.eclipse.jgit.transport.resolver._
import org.slf4j.LoggerFactory
-
import javax.servlet.ServletConfig
-import javax.servlet.ServletContext
-import javax.servlet.http.{HttpServletResponse, HttpServletRequest}
+import javax.servlet.http.{HttpServletRequest, HttpServletResponse}
+
+import org.json4s.jackson.Serialization._
/**
* Provides Git repository via HTTP.
*
* This servlet provides only Git repository functionality.
- * Authentication is provided by [[BasicAuthenticationFilter]].
+ * Authentication is provided by [[GitAuthenticationFilter]].
*/
class GitRepositoryServlet extends GitServlet with SystemSettingsService {
private val logger = LoggerFactory.getLogger(classOf[GitRepositoryServlet])
-
+ private implicit val jsonFormats = gitbucket.core.api.JsonFormat.jsonFormats
+
override def init(config: ServletConfig): Unit = {
setReceivePackFactory(new GitBucketReceivePackFactory())
- // TODO are there any other ways...?
- super.init(new ServletConfig(){
- def getInitParameter(name: String): String = name match {
- case "base-path" => Directory.RepositoryHome
- case "export-all" => "true"
- case name => config.getInitParameter(name)
- }
- def getInitParameterNames(): java.util.Enumeration[String] = {
- config.getInitParameterNames
- }
-
- def getServletContext(): ServletContext = config.getServletContext
- def getServletName(): String = config.getServletName
- })
+ val root: File = new File(Directory.RepositoryHome)
+ setRepositoryResolver(new GitBucketRepositoryResolver(new FileResolver[HttpServletRequest](root, true)))
super.init(config)
}
@@ -56,15 +47,89 @@
override def service(req: HttpServletRequest, res: HttpServletResponse): Unit = {
val agent = req.getHeader("USER-AGENT")
val index = req.getRequestURI.indexOf(".git")
- if(index >= 0 && (agent == null || agent.toLowerCase.indexOf("git/") < 0)){
+ if(index >= 0 && (agent == null || agent.toLowerCase.indexOf("git") < 0)){
// redirect for browsers
val paths = req.getRequestURI.substring(0, index).split("/")
res.sendRedirect(baseUrl(req) + "/" + paths.dropRight(1).last + "/" + paths.last)
+
+ } else if(req.getMethod.toUpperCase == "POST" && req.getRequestURI.endsWith("/info/lfs/objects/batch")){
+ serviceGitLfsBatchAPI(req, res)
+
} else {
// response for git client
super.service(req, res)
}
}
+
+ /**
+ * Provides GitLFS Batch API
+ * https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
+ */
+ protected def serviceGitLfsBatchAPI(req: HttpServletRequest, res: HttpServletResponse): Unit = {
+ val batchRequest = read[GitLfs.BatchRequest](req.getInputStream)
+ val settings = loadSystemSettings()
+
+ settings.baseUrl match {
+ case None => {
+ throw new IllegalStateException("lfs.server_url is not configured.")
+ }
+ case Some(baseUrl) => {
+ req.getRequestURI.substring(1).replace(".git/", "/").split("/") match {
+ case Array(_, owner, repository, _*) => {
+ val timeout = System.currentTimeMillis + (60000 * 10) // 10 min.
+ val batchResponse = batchRequest.operation match {
+ case "upload" =>
+ GitLfs.BatchUploadResponse("basic", batchRequest.objects.map { requestObject =>
+ GitLfs.BatchResponseObject(requestObject.oid, requestObject.size, true,
+ GitLfs.Actions(
+ upload = Some(GitLfs.Action(
+ href = baseUrl + "/git-lfs/" + owner + "/" + repository + "/" + requestObject.oid,
+ header = Map("Authorization" -> StringUtil.encodeBlowfish(timeout + " " + requestObject.oid)),
+ expires_at = new Date(timeout)
+ ))
+ )
+ )
+ })
+ case "download" =>
+ GitLfs.BatchUploadResponse("basic", batchRequest.objects.map { requestObject =>
+ GitLfs.BatchResponseObject(requestObject.oid, requestObject.size, true,
+ GitLfs.Actions(
+ download = Some(GitLfs.Action(
+ href = baseUrl + "/git-lfs/" + owner + "/" + repository + "/" + requestObject.oid,
+ header = Map("Authorization" -> StringUtil.encodeBlowfish(timeout + " " + requestObject.oid)),
+ expires_at = new Date(timeout)
+ ))
+ )
+ )
+ })
+ }
+
+ res.setContentType("application/vnd.git-lfs+json")
+ using(res.getWriter){ out =>
+ out.print(write(batchResponse))
+ out.flush()
+ }
+ }
+ }
+ }
+ }
+ }
+}
+
+class GitBucketRepositoryResolver(parent: FileResolver[HttpServletRequest]) extends RepositoryResolver[HttpServletRequest] {
+
+ private val resolver = new FileResolver[HttpServletRequest](new File(Directory.GitBucketHome), true)
+
+ override def open(req: HttpServletRequest, name: String): Repository = {
+ // Rewrite repository path if routing is marched
+ PluginRegistry().getRepositoryRouting("/" + name).map { case GitRepositoryRouting(urlPattern, localPath, _) =>
+ val path = urlPattern.r.replaceFirstIn(name, localPath)
+ resolver.open(req, path)
+ }.getOrElse {
+ parent.open(req, name)
+ }
+ }
+
}
class GitBucketReceivePackFactory extends ReceivePackFactory[HttpServletRequest] with SystemSettingsService {
@@ -73,148 +138,203 @@
override def create(request: HttpServletRequest, db: Repository): ReceivePack = {
val receivePack = new ReceivePack(db)
- val pusher = request.getAttribute(Keys.Request.UserName).asInstanceOf[String]
- logger.debug("requestURI: " + request.getRequestURI)
- logger.debug("pusher:" + pusher)
+ if(PluginRegistry().getRepositoryRouting(request.gitRepositoryPath).isEmpty){
+ val pusher = request.getAttribute(Keys.Request.UserName).asInstanceOf[String]
- defining(request.paths){ paths =>
- val owner = paths(1)
- val repository = paths(2).stripSuffix(".git")
+ logger.debug("requestURI: " + request.getRequestURI)
+ logger.debug("pusher:" + pusher)
- logger.debug("repository:" + owner + "/" + repository)
+ defining(request.paths){ paths =>
+ val owner = paths(1)
+ val repository = paths(2).stripSuffix(".git")
- if(!repository.endsWith(".wiki")){
- defining(request) { implicit r =>
- val hook = new CommitLogHook(owner, repository, pusher, baseUrl)
- receivePack.setPreReceiveHook(hook)
- receivePack.setPostReceiveHook(hook)
+ logger.debug("repository:" + owner + "/" + repository)
+
+ if(!repository.endsWith(".wiki")){
+ defining(request) { implicit r =>
+ val hook = new CommitLogHook(owner, repository, pusher, baseUrl)
+ receivePack.setPreReceiveHook(hook)
+ receivePack.setPostReceiveHook(hook)
+ }
}
}
- receivePack
}
+
+ receivePack
}
}
import scala.collection.JavaConverters._
-class CommitLogHook(owner: String, repository: String, pusher: String, baseUrl: String)(implicit session: Session)
+class CommitLogHook(owner: String, repository: String, pusher: String, baseUrl: String)/*(implicit session: Session)*/
extends PostReceiveHook with PreReceiveHook
with RepositoryService with AccountService with IssuesService with ActivityService with PullRequestService with WebHookService
- with WebHookPullRequestService {
+ with WebHookPullRequestService with CommitsService {
private val logger = LoggerFactory.getLogger(classOf[CommitLogHook])
private var existIds: Seq[String] = Nil
def onPreReceive(receivePack: ReceivePack, commands: java.util.Collection[ReceiveCommand]): Unit = {
- try {
- using(Git.open(Directory.getRepositoryDir(owner, repository))) { git =>
- existIds = JGitUtil.getAllCommitIds(git)
- }
- } catch {
- case ex: Exception => {
- logger.error(ex.toString, ex)
- throw ex
+ Database() withTransaction { implicit session =>
+ try {
+ commands.asScala.foreach { command =>
+ // call pre-commit hook
+ PluginRegistry().getReceiveHooks
+ .flatMap(_.preReceive(owner, repository, receivePack, command, pusher))
+ .headOption.foreach { error =>
+ command.setResult(ReceiveCommand.Result.REJECTED_OTHER_REASON, error)
+ }
+ }
+ using(Git.open(Directory.getRepositoryDir(owner, repository))) { git =>
+ existIds = JGitUtil.getAllCommitIds(git)
+ }
+ } catch {
+ case ex: Exception => {
+ logger.error(ex.toString, ex)
+ throw ex
+ }
}
}
}
def onPostReceive(receivePack: ReceivePack, commands: java.util.Collection[ReceiveCommand]): Unit = {
- try {
- using(Git.open(Directory.getRepositoryDir(owner, repository))) { git =>
- val pushedIds = scala.collection.mutable.Set[String]()
- commands.asScala.foreach { command =>
- logger.debug(s"commandType: ${command.getType}, refName: ${command.getRefName}")
- implicit val apiContext = api.JsonFormat.Context(baseUrl)
- val refName = command.getRefName.split("/")
- val branchName = refName.drop(2).mkString("/")
- val commits = if (refName(1) == "tags") {
- Nil
- } else {
- command.getType match {
- case ReceiveCommand.Type.DELETE => Nil
- case _ => JGitUtil.getCommitLog(git, command.getOldId.name, command.getNewId.name)
- }
- }
+ Database() withTransaction { implicit session =>
+ try {
+ using(Git.open(Directory.getRepositoryDir(owner, repository))) { git =>
+ JGitUtil.removeCache(git)
- // Retrieve all issue count in the repository
- val issueCount =
- countIssue(IssueSearchCondition(state = "open"), false, owner -> repository) +
- countIssue(IssueSearchCondition(state = "closed"), false, owner -> repository)
-
- val repositoryInfo = getRepository(owner, repository, baseUrl).get
-
- // Extract new commit and apply issue comment
- val defaultBranch = repositoryInfo.repository.defaultBranch
- val newCommits = commits.flatMap { commit =>
- if (!existIds.contains(commit.id) && !pushedIds.contains(commit.id)) {
- if (issueCount > 0) {
- pushedIds.add(commit.id)
- createIssueComment(commit)
- // close issues
- if(refName(1) == "heads" && branchName == defaultBranch && command.getType == ReceiveCommand.Type.UPDATE){
- closeIssuesFromMessage(commit.fullMessage, pusher, owner, repository)
- }
+ val pushedIds = scala.collection.mutable.Set[String]()
+ commands.asScala.foreach { command =>
+ logger.debug(s"commandType: ${command.getType}, refName: ${command.getRefName}")
+ implicit val apiContext = api.JsonFormat.Context(baseUrl)
+ val refName = command.getRefName.split("/")
+ val branchName = refName.drop(2).mkString("/")
+ val commits = if (refName(1) == "tags") {
+ Nil
+ } else {
+ command.getType match {
+ case ReceiveCommand.Type.DELETE => Nil
+ case _ => JGitUtil.getCommitLog(git, command.getOldId.name, command.getNewId.name)
}
- Some(commit)
- } else None
- }
-
- // record activity
- if(refName(1) == "heads"){
- command.getType match {
- case ReceiveCommand.Type.CREATE => recordCreateBranchActivity(owner, repository, pusher, branchName)
- case ReceiveCommand.Type.UPDATE => recordPushActivity(owner, repository, pusher, branchName, newCommits)
- case ReceiveCommand.Type.DELETE => recordDeleteBranchActivity(owner, repository, pusher, branchName)
- case _ =>
}
- } else if(refName(1) == "tags"){
- command.getType match {
- case ReceiveCommand.Type.CREATE => recordCreateTagActivity(owner, repository, pusher, branchName, newCommits)
- case ReceiveCommand.Type.DELETE => recordDeleteTagActivity(owner, repository, pusher, branchName, newCommits)
- case _ =>
- }
- }
- if(refName(1) == "heads"){
- command.getType match {
- case ReceiveCommand.Type.CREATE |
- ReceiveCommand.Type.UPDATE |
- ReceiveCommand.Type.UPDATE_NONFASTFORWARD =>
- updatePullRequests(owner, repository, branchName)
- getAccountByUserName(pusher).map{ pusherAccount =>
- callPullRequestWebHookByRequestBranch("synchronize", repositoryInfo, branchName, baseUrl, pusherAccount)
+ // Retrieve all issue count in the repository
+ val issueCount =
+ countIssue(IssueSearchCondition(state = "open"), false, owner -> repository) +
+ countIssue(IssueSearchCondition(state = "closed"), false, owner -> repository)
+
+ val repositoryInfo = getRepository(owner, repository).get
+
+ // Extract new commit and apply issue comment
+ val defaultBranch = repositoryInfo.repository.defaultBranch
+ val newCommits = commits.flatMap { commit =>
+ if (!existIds.contains(commit.id) && !pushedIds.contains(commit.id)) {
+ if (issueCount > 0) {
+ pushedIds.add(commit.id)
+ createIssueComment(owner, repository, commit)
+ // close issues
+ if (refName(1) == "heads" && branchName == defaultBranch && command.getType == ReceiveCommand.Type.UPDATE) {
+ closeIssuesFromMessage(commit.fullMessage, pusher, owner, repository)
+ }
}
- case _ =>
+ Some(commit)
+ } else None
}
- }
- // call web hook
- callWebHookOf(owner, repository, "push"){
- for(pusherAccount <- getAccountByUserName(pusher);
- ownerAccount <- getAccountByUserName(owner)) yield {
- WebHookPushPayload(git, pusherAccount, command.getRefName, repositoryInfo, newCommits, ownerAccount)
+ // record activity
+ if (refName(1) == "heads") {
+ command.getType match {
+ case ReceiveCommand.Type.CREATE => recordCreateBranchActivity(owner, repository, pusher, branchName)
+ case ReceiveCommand.Type.UPDATE => recordPushActivity(owner, repository, pusher, branchName, newCommits)
+ case ReceiveCommand.Type.DELETE => recordDeleteBranchActivity(owner, repository, pusher, branchName)
+ case _ =>
+ }
+ } else if (refName(1) == "tags") {
+ command.getType match {
+ case ReceiveCommand.Type.CREATE => recordCreateTagActivity(owner, repository, pusher, branchName, newCommits)
+ case ReceiveCommand.Type.DELETE => recordDeleteTagActivity(owner, repository, pusher, branchName, newCommits)
+ case _ =>
+ }
}
+
+ if (refName(1) == "heads") {
+ command.getType match {
+ case ReceiveCommand.Type.CREATE |
+ ReceiveCommand.Type.UPDATE |
+ ReceiveCommand.Type.UPDATE_NONFASTFORWARD =>
+ updatePullRequests(owner, repository, branchName)
+ getAccountByUserName(pusher).map { pusherAccount =>
+ callPullRequestWebHookByRequestBranch("synchronize", repositoryInfo, branchName, baseUrl, pusherAccount)
+ }
+ case _ =>
+ }
+ }
+
+ // call web hook
+ callWebHookOf(owner, repository, WebHook.Push) {
+ for (pusherAccount <- getAccountByUserName(pusher);
+ ownerAccount <- getAccountByUserName(owner)) yield {
+ WebHookPushPayload(git, pusherAccount, command.getRefName, repositoryInfo, newCommits, ownerAccount,
+ newId = command.getNewId(), oldId = command.getOldId())
+ }
+ }
+
+ // call post-commit hook
+ PluginRegistry().getReceiveHooks.foreach(_.postReceive(owner, repository, receivePack, command, pusher))
}
}
- }
- // update repository last modified time.
- updateLastActivityDate(owner, repository)
- } catch {
- case ex: Exception => {
- logger.error(ex.toString, ex)
- throw ex
- }
- }
- }
-
- private def createIssueComment(commit: CommitInfo) = {
- StringUtil.extractIssueId(commit.fullMessage).foreach { issueId =>
- if(getIssue(owner, repository, issueId).isDefined){
- getAccountByMailAddress(commit.committerEmailAddress).foreach { account =>
- createComment(owner, repository, account.userName, issueId.toInt, commit.fullMessage + " " + commit.id, "commit")
+ // update repository last modified time.
+ updateLastActivityDate(owner, repository)
+ } catch {
+ case ex: Exception => {
+ logger.error(ex.toString, ex)
+ throw ex
}
}
}
}
+
+}
+
+object GitLfs {
+
+ case class BatchRequest(
+ operation: String,
+ transfers: Seq[String],
+ objects: Seq[BatchRequestObject]
+ )
+
+ case class BatchRequestObject(
+ oid: String,
+ size: Long
+ )
+
+ case class BatchUploadResponse(
+ transfer: String,
+ objects: Seq[BatchResponseObject]
+ )
+
+ case class BatchResponseObject(
+ oid: String,
+ size: Long,
+ authenticated: Boolean,
+ actions: Actions
+ )
+
+ case class Actions(
+ download: Option[Action] = None,
+ upload: Option[Action] = None
+ )
+
+ case class Action(
+ href: String,
+ header: Map[String, String] = Map.empty,
+ expires_at: Date
+ )
+
+ case class Error(
+ message: String
+ )
+
}
diff --git a/src/main/scala/gitbucket/core/servlet/InitializeListener.scala b/src/main/scala/gitbucket/core/servlet/InitializeListener.scala
index a375b37..8d55a81 100644
--- a/src/main/scala/gitbucket/core/servlet/InitializeListener.scala
+++ b/src/main/scala/gitbucket/core/servlet/InitializeListener.scala
@@ -1,16 +1,23 @@
package gitbucket.core.servlet
+import java.io.File
+
import akka.event.Logging
import com.typesafe.config.ConfigFactory
+import gitbucket.core.GitBucketCoreModule
import gitbucket.core.plugin.PluginRegistry
import gitbucket.core.service.{ActivityService, SystemSettingsService}
-import org.apache.commons.io.FileUtils
+import gitbucket.core.util.DatabaseConfig
+import gitbucket.core.util.Directory._
+import gitbucket.core.util.JDBCUtil._
+import io.github.gitbucket.solidbase.Solidbase
+import io.github.gitbucket.solidbase.manager.JDBCVersionManager
import javax.servlet.{ServletContextListener, ServletContextEvent}
+import org.apache.commons.io.FileUtils
import org.slf4j.LoggerFactory
-import gitbucket.core.util.Versions
import akka.actor.{Actor, Props, ActorSystem}
import com.typesafe.akka.extension.quartz.QuartzSchedulerExtension
-import AutoUpdate._
+import scala.collection.JavaConverters._
/**
* Initialize GitBucket system.
@@ -29,15 +36,63 @@
Database() withTransaction { session =>
val conn = session.conn
+ val manager = new JDBCVersionManager(conn)
- // Migration
- logger.debug("Start schema update")
- Versions.update(conn, headVersion, getCurrentVersion(), versions, Thread.currentThread.getContextClassLoader){ conn =>
- FileUtils.writeStringToFile(versionFile, headVersion.versionString, "UTF-8")
+ // Check version
+ val versionFile = new File(GitBucketHome, "version")
+
+ if(versionFile.exists()){
+ val version = FileUtils.readFileToString(versionFile, "UTF-8")
+ if(version == "3.14"){
+ // Initialization for GitBucket 3.14
+ logger.info("Migration to GitBucket 4.x start")
+
+ // Backup current data
+ val dataMvFile = new File(GitBucketHome, "data.mv.db")
+ if(dataMvFile.exists) {
+ FileUtils.copyFile(dataMvFile, new File(GitBucketHome, "data.mv.db_3.14"))
+ }
+ val dataTraceFile = new File(GitBucketHome, "data.trace.db")
+ if(dataTraceFile.exists) {
+ FileUtils.copyFile(dataTraceFile, new File(GitBucketHome, "data.trace.db_3.14"))
+ }
+
+ // Change form
+ manager.initialize()
+ manager.updateVersion(GitBucketCoreModule.getModuleId, "4.0.0")
+ conn.select("SELECT PLUGIN_ID, VERSION FROM PLUGIN"){ rs =>
+ manager.updateVersion(rs.getString("PLUGIN_ID"), rs.getString("VERSION"))
+ }
+ conn.update("DROP TABLE PLUGIN")
+ versionFile.delete()
+
+ logger.info("Migration to GitBucket 4.x completed")
+
+ } else {
+ throw new Exception("GitBucket can't migrate from this version. Please update to 3.14 at first.")
+ }
+ }
+
+ // Run normal migration
+ logger.info("Start schema update")
+ val solidbase = new Solidbase()
+ solidbase.migrate(conn, Thread.currentThread.getContextClassLoader, DatabaseConfig.liquiDriver, GitBucketCoreModule)
+
+ // Rescue code for users who updated from 3.14 to 4.0.0
+ // https://github.com/gitbucket/gitbucket/issues/1227
+ val currentVersion = manager.getCurrentVersion(GitBucketCoreModule.getModuleId)
+ val databaseVersion = if(currentVersion == "4.0"){
+ manager.updateVersion(GitBucketCoreModule.getModuleId, "4.0.0")
+ "4.0.0"
+ } else currentVersion
+
+ val gitbucketVersion = GitBucketCoreModule.getVersions.asScala.last.getVersion
+ if(databaseVersion != gitbucketVersion){
+ throw new IllegalStateException(s"Initialization failed. GitBucket version is ${gitbucketVersion}, but database version is ${databaseVersion}.")
}
// Load plugins
- logger.debug("Initialize plugins")
+ logger.info("Initialize plugins")
PluginRegistry.initialize(event.getServletContext, loadSystemSettings(), conn)
}
diff --git a/src/main/scala/gitbucket/core/servlet/PluginAssetsServlet.scala b/src/main/scala/gitbucket/core/servlet/PluginAssetsServlet.scala
new file mode 100644
index 0000000..0d43af3
--- /dev/null
+++ b/src/main/scala/gitbucket/core/servlet/PluginAssetsServlet.scala
@@ -0,0 +1,39 @@
+package gitbucket.core.servlet
+
+import javax.servlet.http.{HttpServlet, HttpServletRequest, HttpServletResponse}
+
+import gitbucket.core.plugin.PluginRegistry
+import gitbucket.core.util.FileUtil
+import org.apache.commons.io.IOUtils
+
+/**
+ * Supply assets which are provided by plugins.
+ */
+class PluginAssetsServlet extends HttpServlet {
+
+ override def doGet(req: HttpServletRequest, resp: HttpServletResponse): Unit = {
+ val assetsMappings = PluginRegistry().getAssetsMappings
+ val path = req.getRequestURI.substring(req.getContextPath.length)
+
+ assetsMappings
+ .find { case (prefix, _, _) => path.startsWith("/plugin-assets" + prefix) }
+ .flatMap { case (prefix, resourcePath, classLoader) =>
+ val resourceName = path.substring(("/plugin-assets" + prefix).length)
+ Option(classLoader.getResourceAsStream(resourcePath.replaceFirst("^/", "") + resourceName))
+ }
+ .map { in =>
+ try {
+ val bytes = IOUtils.toByteArray(in)
+ resp.setContentLength(bytes.length)
+ resp.setContentType(FileUtil.getContentType(path, bytes))
+ resp.getOutputStream.write(bytes)
+ } finally {
+ in.close()
+ }
+ }
+ .getOrElse {
+ resp.setStatus(404)
+ }
+ }
+
+}
diff --git a/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala b/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala
index fc2e457..f4dc1b9 100644
--- a/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala
+++ b/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala
@@ -2,7 +2,7 @@
import javax.servlet._
import javax.servlet.http.HttpServletRequest
-import com.mchange.v2.c3p0.ComboPooledDataSource
+import com.zaxxer.hikari._
import gitbucket.core.util.DatabaseConfig
import org.scalatra.ScalatraBase
import org.slf4j.LoggerFactory
@@ -21,8 +21,9 @@
def destroy(): Unit = {}
def doFilter(req: ServletRequest, res: ServletResponse, chain: FilterChain): Unit = {
- if(req.asInstanceOf[HttpServletRequest].getServletPath().startsWith("/assets/")){
- // assets don't need transaction
+ val servletPath = req.asInstanceOf[HttpServletRequest].getServletPath()
+ if(servletPath.startsWith("/assets/") || servletPath == "/git" || servletPath == "/git-lfs"){
+ // assets and git-lfs don't need transaction
chain.doFilter(req, res)
} else {
Database() withTransaction { session =>
@@ -46,14 +47,21 @@
private val logger = LoggerFactory.getLogger(Database.getClass)
- private val dataSource: ComboPooledDataSource = {
- val ds = new ComboPooledDataSource
- ds.setDriverClass(DatabaseConfig.driver)
- ds.setJdbcUrl(DatabaseConfig.url)
- ds.setUser(DatabaseConfig.user)
- ds.setPassword(DatabaseConfig.password)
+ private val dataSource: HikariDataSource = {
+ val config = new HikariConfig()
+ config.setDriverClassName(DatabaseConfig.jdbcDriver)
+ config.setJdbcUrl(DatabaseConfig.url)
+ config.setUsername(DatabaseConfig.user)
+ config.setPassword(DatabaseConfig.password)
+ config.setAutoCommit(false)
+ DatabaseConfig.connectionTimeout.foreach(config.setConnectionTimeout)
+ DatabaseConfig.idleTimeout.foreach(config.setIdleTimeout)
+ DatabaseConfig.maxLifetime.foreach(config.setMaxLifetime)
+ DatabaseConfig.minimumIdle.foreach(config.setMinimumIdle)
+ DatabaseConfig.maximumPoolSize.foreach(config.setMaximumPoolSize)
+
logger.debug("load database connection pool")
- ds
+ new HikariDataSource(config)
}
private val db: SlickDatabase = {
diff --git a/src/main/scala/gitbucket/core/ssh/GitCommand.scala b/src/main/scala/gitbucket/core/ssh/GitCommand.scala
index c78ff82..d5d8135 100644
--- a/src/main/scala/gitbucket/core/ssh/GitCommand.scala
+++ b/src/main/scala/gitbucket/core/ssh/GitCommand.scala
@@ -1,55 +1,62 @@
package gitbucket.core.ssh
import gitbucket.core.model.Session
+import gitbucket.core.plugin.{GitRepositoryRouting, PluginRegistry}
import gitbucket.core.service.{RepositoryService, AccountService, SystemSettingsService}
import gitbucket.core.servlet.{Database, CommitLogHook}
import gitbucket.core.util.{Directory, ControlUtil}
-import org.apache.sshd.server.{CommandFactory, Environment, ExitCallback, Command}
+import org.apache.sshd.server.{CommandFactory, Environment, ExitCallback, Command, SessionAware}
+import org.apache.sshd.server.session.ServerSession
import org.slf4j.LoggerFactory
-import java.io.{InputStream, OutputStream}
+import java.io.{File, InputStream, OutputStream}
import ControlUtil._
import org.eclipse.jgit.api.Git
import Directory._
import org.eclipse.jgit.transport.{ReceivePack, UploadPack}
-import org.apache.sshd.server.command.UnknownCommand
+import org.apache.sshd.server.scp.UnknownCommand
import org.eclipse.jgit.errors.RepositoryNotFoundException
object GitCommand {
- val CommandRegex = """\Agit-(upload|receive)-pack '/([a-zA-Z0-9\-_.]+)/([a-zA-Z0-9\-_.]+).git'\Z""".r
+ val DefaultCommandRegex = """\Agit-(upload|receive)-pack '/([a-zA-Z0-9\-_.]+)/([a-zA-Z0-9\-_.]+).git'\Z""".r
+ val SimpleCommandRegex = """\Agit-(upload|receive)-pack '/(.+\.git)'\Z""".r
}
-abstract class GitCommand(val owner: String, val repoName: String) extends Command {
- self: RepositoryService with AccountService =>
+abstract class GitCommand extends Command with SessionAware {
private val logger = LoggerFactory.getLogger(classOf[GitCommand])
- protected var err: OutputStream = null
- protected var in: InputStream = null
- protected var out: OutputStream = null
- protected var callback: ExitCallback = null
+ @volatile protected var err: OutputStream = null
+ @volatile protected var in: InputStream = null
+ @volatile protected var out: OutputStream = null
+ @volatile protected var callback: ExitCallback = null
+ @volatile private var authUser:Option[String] = None
- protected def runTask(user: String)(implicit session: Session): Unit
+ protected def runTask(authUser: String): Unit
- private def newTask(user: String): Runnable = new Runnable {
+ private def newTask(): Runnable = new Runnable {
override def run(): Unit = {
- Database() withSession { implicit session =>
- try {
- runTask(user)
- callback.onExit(0)
- } catch {
- case e: RepositoryNotFoundException =>
- logger.info(e.getMessage)
- callback.onExit(1, "Repository Not Found")
- case e: Throwable =>
- logger.error(e.getMessage, e)
- callback.onExit(1)
- }
+ authUser match {
+ case Some(authUser) =>
+ try {
+ runTask(authUser)
+ callback.onExit(0)
+ } catch {
+ case e: RepositoryNotFoundException =>
+ logger.info(e.getMessage)
+ callback.onExit(1, "Repository Not Found")
+ case e: Throwable =>
+ logger.error(e.getMessage, e)
+ callback.onExit(1)
+ }
+ case None =>
+ val message = "User not authenticated"
+ logger.error(message)
+ callback.onExit(1, message)
}
}
}
- override def start(env: Environment): Unit = {
- val user = env.getEnv.get("USER")
- val thread = new Thread(newTask(user))
+ final override def start(env: Environment): Unit = {
+ val thread = new Thread(newTask())
thread.start()
}
@@ -71,63 +78,125 @@
this.in = in
}
+ override def setSession(serverSession:ServerSession) {
+ this.authUser = PublicKeyAuthenticator.getUserName(serverSession)
+ }
+
+}
+
+abstract class DefaultGitCommand(val owner: String, val repoName: String) extends GitCommand {
+ self: RepositoryService with AccountService =>
+
protected def isWritableUser(username: String, repositoryInfo: RepositoryService.RepositoryInfo)
(implicit session: Session): Boolean =
getAccountByUserName(username) match {
- case Some(account) => hasWritePermission(repositoryInfo.owner, repositoryInfo.name, Some(account))
+ case Some(account) => hasDeveloperRole(repositoryInfo.owner, repositoryInfo.name, Some(account))
case None => false
}
}
-class GitUploadPack(owner: String, repoName: String, baseUrl: String) extends GitCommand(owner, repoName)
+
+class DefaultGitUploadPack(owner: String, repoName: String) extends DefaultGitCommand(owner, repoName)
with RepositoryService with AccountService {
- override protected def runTask(user: String)(implicit session: Session): Unit = {
- getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", ""), baseUrl).foreach { repositoryInfo =>
- if(!repositoryInfo.repository.isPrivate || isWritableUser(user, repositoryInfo)){
- using(Git.open(getRepositoryDir(owner, repoName))) { git =>
- val repository = git.getRepository
- val upload = new UploadPack(repository)
- upload.upload(in, out, err)
- }
+ override protected def runTask(user: String): Unit = {
+ val execute = Database() withSession { implicit session =>
+ getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", "")).map { repositoryInfo =>
+ !repositoryInfo.repository.isPrivate || isWritableUser(user, repositoryInfo)
+ }.getOrElse(false)
+ }
+
+ if(execute){
+ using(Git.open(getRepositoryDir(owner, repoName))) { git =>
+ val repository = git.getRepository
+ val upload = new UploadPack(repository)
+ upload.upload(in, out, err)
}
}
}
-
}
-class GitReceivePack(owner: String, repoName: String, baseUrl: String) extends GitCommand(owner, repoName)
- with SystemSettingsService with RepositoryService with AccountService {
+class DefaultGitReceivePack(owner: String, repoName: String, baseUrl: String) extends DefaultGitCommand(owner, repoName)
+ with RepositoryService with AccountService {
- override protected def runTask(user: String)(implicit session: Session): Unit = {
- getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", ""), baseUrl).foreach { repositoryInfo =>
- if(isWritableUser(user, repositoryInfo)){
- using(Git.open(getRepositoryDir(owner, repoName))) { git =>
- val repository = git.getRepository
- val receive = new ReceivePack(repository)
- if(!repoName.endsWith(".wiki")){
- val hook = new CommitLogHook(owner, repoName, user, baseUrl)
- receive.setPreReceiveHook(hook)
- receive.setPostReceiveHook(hook)
- }
- receive.receive(in, out, err)
+ override protected def runTask(user: String): Unit = {
+ val execute = Database() withSession { implicit session =>
+ getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", "")).map { repositoryInfo =>
+ isWritableUser(user, repositoryInfo)
+ }.getOrElse(false)
+ }
+
+ if(execute) {
+ using(Git.open(getRepositoryDir(owner, repoName))) { git =>
+ val repository = git.getRepository
+ val receive = new ReceivePack(repository)
+ if (!repoName.endsWith(".wiki")) {
+ val hook = new CommitLogHook(owner, repoName, user, baseUrl)
+ receive.setPreReceiveHook(hook)
+ receive.setPostReceiveHook(hook)
}
+ receive.receive(in, out, err)
}
}
}
-
}
+class PluginGitUploadPack(repoName: String, routing: GitRepositoryRouting) extends GitCommand
+ with SystemSettingsService {
+
+ override protected def runTask(user: String): Unit = {
+ val execute = Database() withSession { implicit session =>
+ routing.filter.filter("/" + repoName, Some(user), loadSystemSettings(), false)
+ }
+
+ if(execute){
+ val path = routing.urlPattern.r.replaceFirstIn(repoName, routing.localPath)
+ using(Git.open(new File(Directory.GitBucketHome, path))){ git =>
+ val repository = git.getRepository
+ val upload = new UploadPack(repository)
+ upload.upload(in, out, err)
+ }
+ }
+ }
+}
+
+class PluginGitReceivePack(repoName: String, routing: GitRepositoryRouting) extends GitCommand
+ with SystemSettingsService {
+
+ override protected def runTask(user: String): Unit = {
+ val execute = Database() withSession { implicit session =>
+ routing.filter.filter("/" + repoName, Some(user), loadSystemSettings(), true)
+ }
+ if(execute){
+ val path = routing.urlPattern.r.replaceFirstIn(repoName, routing.localPath)
+ using(Git.open(new File(Directory.GitBucketHome, path))){ git =>
+ val repository = git.getRepository
+ val receive = new ReceivePack(repository)
+ receive.receive(in, out, err)
+ }
+ }
+ }
+}
+
+
class GitCommandFactory(baseUrl: String) extends CommandFactory {
private val logger = LoggerFactory.getLogger(classOf[GitCommandFactory])
override def createCommand(command: String): Command = {
+ import GitCommand._
logger.debug(s"command: $command")
+
command match {
- case GitCommand.CommandRegex("upload", owner, repoName) => new GitUploadPack(owner, repoName, baseUrl)
- case GitCommand.CommandRegex("receive", owner, repoName) => new GitReceivePack(owner, repoName, baseUrl)
+ case SimpleCommandRegex ("upload" , repoName) if(pluginRepository(repoName)) => new PluginGitUploadPack (repoName, routing(repoName))
+ case SimpleCommandRegex ("receive", repoName) if(pluginRepository(repoName)) => new PluginGitReceivePack(repoName, routing(repoName))
+ case DefaultCommandRegex("upload" , owner, repoName) => new DefaultGitUploadPack (owner, repoName)
+ case DefaultCommandRegex("receive", owner, repoName) => new DefaultGitReceivePack(owner, repoName, baseUrl)
case _ => new UnknownCommand(command)
}
}
+
+ private def pluginRepository(repoName: String): Boolean = PluginRegistry().getRepositoryRouting("/" + repoName).isDefined
+ private def routing(repoName: String): GitRepositoryRouting = PluginRegistry().getRepositoryRouting("/" + repoName).get
+
}
diff --git a/src/main/scala/gitbucket/core/ssh/NoShell.scala b/src/main/scala/gitbucket/core/ssh/NoShell.scala
index bd30ccf..065bee0 100644
--- a/src/main/scala/gitbucket/core/ssh/NoShell.scala
+++ b/src/main/scala/gitbucket/core/ssh/NoShell.scala
@@ -1,12 +1,12 @@
package gitbucket.core.ssh
-import gitbucket.core.service.SystemSettingsService
+import gitbucket.core.service.SystemSettingsService.SshAddress
import org.apache.sshd.common.Factory
import org.apache.sshd.server.{Environment, ExitCallback, Command}
import java.io.{OutputStream, InputStream}
import org.eclipse.jgit.lib.Constants
-class NoShell extends Factory[Command] with SystemSettingsService {
+class NoShell(sshAddress:SshAddress) extends Factory[Command] {
override def create(): Command = new Command() {
private var in: InputStream = null
private var out: OutputStream = null
@@ -14,8 +14,6 @@
private var callback: ExitCallback = null
override def start(env: Environment): Unit = {
- val user = env.getEnv.get("USER")
- val port = loadSystemSettings().sshPort.getOrElse(SystemSettingsService.DefaultSshPort)
val message =
"""
| Welcome to
@@ -31,8 +29,8 @@
|
| Please use:
|
- | git clone ssh://%s@GITBUCKET_HOST:%d/OWNER/REPOSITORY_NAME.git
- """.stripMargin.format(user, port).replace("\n", "\r\n") + "\r\n"
+ | git clone ssh://%s@%s:%d/OWNER/REPOSITORY_NAME.git
+ """.stripMargin.format(sshAddress.genericUser, sshAddress.host, sshAddress.port).replace("\n", "\r\n") + "\r\n"
err.write(Constants.encode(message))
err.flush()
in.close()
diff --git a/src/main/scala/gitbucket/core/ssh/PublicKeyAuthenticator.scala b/src/main/scala/gitbucket/core/ssh/PublicKeyAuthenticator.scala
index daf5c30..cc7a905 100644
--- a/src/main/scala/gitbucket/core/ssh/PublicKeyAuthenticator.scala
+++ b/src/main/scala/gitbucket/core/ssh/PublicKeyAuthenticator.scala
@@ -1,22 +1,73 @@
package gitbucket.core.ssh
-import gitbucket.core.service.SshKeyService
-import gitbucket.core.servlet.Database
-import org.apache.sshd.server.PublickeyAuthenticator
-import org.apache.sshd.server.session.ServerSession
import java.security.PublicKey
-class PublicKeyAuthenticator extends PublickeyAuthenticator with SshKeyService {
+import gitbucket.core.service.SshKeyService
+import gitbucket.core.servlet.Database
+import org.apache.sshd.server.auth.pubkey.PublickeyAuthenticator
+import org.apache.sshd.server.session.ServerSession
+import org.apache.sshd.common.AttributeStore
+import org.slf4j.LoggerFactory
- override def authenticate(username: String, key: PublicKey, session: ServerSession): Boolean = {
- Database() withSession { implicit session =>
- getPublicKeys(username).exists { sshKey =>
- SshUtil.str2PublicKey(sshKey.publicKey) match {
- case Some(publicKey) => key.equals(publicKey)
- case _ => false
- }
- }
+object PublicKeyAuthenticator {
+ // put in the ServerSession here to be read by GitCommand later
+ private val userNameSessionKey = new AttributeStore.AttributeKey[String]
+
+ def putUserName(serverSession:ServerSession, userName:String):Unit =
+ serverSession.setAttribute(userNameSessionKey, userName)
+
+ def getUserName(serverSession:ServerSession):Option[String] =
+ Option(serverSession.getAttribute(userNameSessionKey))
+}
+
+class PublicKeyAuthenticator(genericUser:String) extends PublickeyAuthenticator with SshKeyService {
+ private val logger = LoggerFactory.getLogger(classOf[PublicKeyAuthenticator])
+
+ override def authenticate(username: String, key: PublicKey, session: ServerSession): Boolean =
+ if (username == genericUser) authenticateGenericUser(username, key, session, genericUser)
+ else authenticateLoginUser(username, key, session)
+
+ private def authenticateLoginUser(username: String, key: PublicKey, session: ServerSession): Boolean = {
+ val authenticated =
+ Database()
+ .withSession { implicit dbSession => getPublicKeys(username) }
+ .map(_.publicKey)
+ .flatMap(SshUtil.str2PublicKey)
+ .contains(key)
+ if (authenticated) {
+ logger.info(s"authentication as ssh user ${username} succeeded")
+ PublicKeyAuthenticator.putUserName(session, username)
}
+ else {
+ logger.info(s"authentication as ssh user ${username} failed")
+ }
+ authenticated
}
+ private def authenticateGenericUser(username: String, key: PublicKey, session: ServerSession, genericUser:String): Boolean = {
+ // find all users having the key we got from ssh
+ val possibleUserNames =
+ Database()
+ .withSession { implicit dbSession => getAllKeys() }
+ .filter { sshKey =>
+ SshUtil.str2PublicKey(sshKey.publicKey).exists(_ == key)
+ }
+ .map(_.userName)
+ .distinct
+ // determine the user - if different accounts share the same key, tough luck
+ val uniqueUserName =
+ possibleUserNames match {
+ case List() =>
+ logger.info(s"authentication as generic user ${genericUser} failed, public key not found")
+ None
+ case List(name) =>
+ logger.info(s"authentication as generic user ${genericUser} succeeded, identified ${name}")
+ Some(name)
+ case _ =>
+ logger.info(s"authentication as generic user ${genericUser} failed, public key is ambiguous")
+ None
+ }
+ uniqueUserName.foreach(PublicKeyAuthenticator.putUserName(session, _))
+ uniqueUserName.isDefined
+ }
}
diff --git a/src/main/scala/gitbucket/core/ssh/SshServerListener.scala b/src/main/scala/gitbucket/core/ssh/SshServerListener.scala
index 1288eb1..6d3c123 100644
--- a/src/main/scala/gitbucket/core/ssh/SshServerListener.scala
+++ b/src/main/scala/gitbucket/core/ssh/SshServerListener.scala
@@ -1,28 +1,34 @@
package gitbucket.core.ssh
+import java.io.File
+import java.util.concurrent.atomic.AtomicBoolean
import javax.servlet.{ServletContextEvent, ServletContextListener}
+
import gitbucket.core.service.SystemSettingsService
-import gitbucket.core.util.Directory
+import gitbucket.core.service.SystemSettingsService.SshAddress
+import gitbucket.core.util.{Directory}
import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider
import org.slf4j.LoggerFactory
-import java.util.concurrent.atomic.AtomicBoolean
object SshServer {
private val logger = LoggerFactory.getLogger(SshServer.getClass)
- private val server = org.apache.sshd.SshServer.setUpDefaultServer()
+ private val server = org.apache.sshd.server.SshServer.setUpDefaultServer()
private val active = new AtomicBoolean(false)
- private def configure(port: Int, baseUrl: String) = {
- server.setPort(port)
- server.setKeyPairProvider(new SimpleGeneratorHostKeyProvider(s"${Directory.GitBucketHome}/gitbucket.ser"))
- server.setPublickeyAuthenticator(new PublicKeyAuthenticator)
+ private def configure(sshAddress: SshAddress, baseUrl: String) = {
+ server.setPort(sshAddress.port)
+ val provider = new SimpleGeneratorHostKeyProvider(new File(s"${Directory.GitBucketHome}/gitbucket.ser"))
+ provider.setAlgorithm("RSA")
+ provider.setOverwriteAllowed(false)
+ server.setKeyPairProvider(provider)
+ server.setPublickeyAuthenticator(new PublicKeyAuthenticator(sshAddress.genericUser))
server.setCommandFactory(new GitCommandFactory(baseUrl))
- server.setShellFactory(new NoShell)
+ server.setShellFactory(new NoShell(sshAddress))
}
- def start(port: Int, baseUrl: String) = {
+ def start(sshAddress: SshAddress, baseUrl: String) = {
if(active.compareAndSet(false, true)){
- configure(port, baseUrl)
+ configure(sshAddress, baseUrl)
server.start()
logger.info(s"Start SSH Server Listen on ${server.getPort}")
}
@@ -50,20 +56,18 @@
override def contextInitialized(sce: ServletContextEvent): Unit = {
val settings = loadSystemSettings()
- if(settings.ssh){
- settings.baseUrl match {
- case None =>
- logger.error("Could not start SshServer because the baseUrl is not configured.")
- case Some(baseUrl) =>
- SshServer.start(settings.sshPort.getOrElse(SystemSettingsService.DefaultSshPort), baseUrl)
- }
+ if (settings.sshAddress.isDefined && settings.baseUrl.isEmpty) {
+ logger.error("Could not start SshServer because the baseUrl is not configured.")
}
+ for {
+ sshAddress <- settings.sshAddress
+ baseUrl <- settings.baseUrl
+ }
+ SshServer.start(sshAddress, baseUrl)
}
override def contextDestroyed(sce: ServletContextEvent): Unit = {
- if(loadSystemSettings().ssh){
- SshServer.stop()
- }
+ SshServer.stop()
}
}
diff --git a/src/main/scala/gitbucket/core/ssh/SshUtil.scala b/src/main/scala/gitbucket/core/ssh/SshUtil.scala
index 512332c..42167ed 100644
--- a/src/main/scala/gitbucket/core/ssh/SshUtil.scala
+++ b/src/main/scala/gitbucket/core/ssh/SshUtil.scala
@@ -1,10 +1,13 @@
package gitbucket.core.ssh
import java.security.PublicKey
-import org.slf4j.LoggerFactory
+
import org.apache.commons.codec.binary.Base64
+import org.apache.sshd.common.config.keys.KeyUtils
+import org.apache.sshd.common.util.buffer.ByteArrayBuffer
import org.eclipse.jgit.lib.Constants
-import org.apache.sshd.common.util.{KeyUtils, Buffer}
+import org.slf4j.LoggerFactory
+
object SshUtil {
@@ -20,7 +23,7 @@
try {
val encodedKey = parts(1)
val decode = Base64.decodeBase64(Constants.encodeASCII(encodedKey))
- Some(new Buffer(decode).getRawPublicKey)
+ Some(new ByteArrayBuffer(decode).getRawPublicKey)
} catch {
case e: Throwable =>
logger.debug(e.getMessage, e)
@@ -28,9 +31,7 @@
}
}
- def fingerPrint(key: String): Option[String] = str2PublicKey(key) match {
- case Some(publicKey) => Some(KeyUtils.getFingerPrint(publicKey))
- case None => None
- }
+ def fingerPrint(key: String): Option[String] =
+ str2PublicKey(key) map KeyUtils.getFingerPrint
}
diff --git a/src/main/scala/gitbucket/core/util/AuthUtil.scala b/src/main/scala/gitbucket/core/util/AuthUtil.scala
new file mode 100644
index 0000000..7a83c13
--- /dev/null
+++ b/src/main/scala/gitbucket/core/util/AuthUtil.scala
@@ -0,0 +1,21 @@
+package gitbucket.core.util
+
+import javax.servlet.http.HttpServletResponse
+
+/**
+ * Provides HTTP (Basic) Authentication related functions.
+ */
+object AuthUtil {
+ def requireAuth(response: HttpServletResponse): Unit = {
+ response.setHeader("WWW-Authenticate", "BASIC realm=\"GitBucket\"")
+ response.sendError(HttpServletResponse.SC_UNAUTHORIZED)
+ }
+
+ def decodeAuthHeader(header: String): String = {
+ try {
+ new String(new sun.misc.BASE64Decoder().decodeBuffer(header.substring(6)))
+ } catch {
+ case _: Throwable => ""
+ }
+ }
+}
\ No newline at end of file
diff --git a/src/main/scala/gitbucket/core/util/Authenticator.scala b/src/main/scala/gitbucket/core/util/Authenticator.scala
index 49de665..bbcb339 100644
--- a/src/main/scala/gitbucket/core/util/Authenticator.scala
+++ b/src/main/scala/gitbucket/core/util/Authenticator.scala
@@ -1,7 +1,8 @@
package gitbucket.core.util
import gitbucket.core.controller.ControllerBase
-import gitbucket.core.service.{RepositoryService, AccountService}
+import gitbucket.core.service.{AccountService, RepositoryService}
+import gitbucket.core.model.Role
import RepositoryService.RepositoryInfo
import Implicits._
import ControlUtil._
@@ -36,13 +37,13 @@
private def authenticate(action: (RepositoryInfo) => Any) = {
{
defining(request.paths){ paths =>
- getRepository(paths(0), paths(1), baseUrl).map { repository =>
+ getRepository(paths(0), paths(1)).map { repository =>
context.loginAccount match {
case Some(x) if(x.isAdmin) => action(repository)
case Some(x) if(repository.owner == x.userName) => action(repository)
- case Some(x) if(getGroupMembers(repository.owner).exists { member =>
- member.userName == x.userName && member.isManager == true
- }) => action(repository)
+ // TODO Repository management is allowed for only group managers?
+ case Some(x) if(getGroupMembers(repository.owner).exists { m => m.userName == x.userName && m.isManager == true }) => action(repository)
+ case Some(x) if(getCollaboratorUserNames(paths(0), paths(1), Seq(Role.ADMIN)).contains(x.userName)) => action(repository)
case _ => Unauthorized()
}
} getOrElse NotFound()
@@ -86,46 +87,24 @@
}
/**
- * Allows only collaborators and administrators.
+ * Allows only guests and signed in users who can access the repository.
*/
-trait CollaboratorsAuthenticator { self: ControllerBase with RepositoryService =>
- protected def collaboratorsOnly(action: (RepositoryInfo) => Any) = { authenticate(action) }
- protected def collaboratorsOnly[T](action: (T, RepositoryInfo) => Any) = (form: T) => { authenticate(action(form, _)) }
-
- private def authenticate(action: (RepositoryInfo) => Any) = {
- {
- defining(request.paths){ paths =>
- getRepository(paths(0), paths(1), baseUrl).map { repository =>
- context.loginAccount match {
- case Some(x) if(x.isAdmin) => action(repository)
- case Some(x) if(paths(0) == x.userName) => action(repository)
- case Some(x) if(getCollaborators(paths(0), paths(1)).contains(x.userName)) => action(repository)
- case _ => Unauthorized()
- }
- } getOrElse NotFound()
- }
- }
- }
-}
-
-/**
- * Allows only the repository owner (or manager for group repository) and administrators.
- */
-trait ReferrerAuthenticator { self: ControllerBase with RepositoryService =>
+trait ReferrerAuthenticator { self: ControllerBase with RepositoryService with AccountService =>
protected def referrersOnly(action: (RepositoryInfo) => Any) = { authenticate(action) }
protected def referrersOnly[T](action: (T, RepositoryInfo) => Any) = (form: T) => { authenticate(action(form, _)) }
private def authenticate(action: (RepositoryInfo) => Any) = {
{
defining(request.paths){ paths =>
- getRepository(paths(0), paths(1), baseUrl).map { repository =>
+ getRepository(paths(0), paths(1)).map { repository =>
if(!repository.repository.isPrivate){
action(repository)
} else {
context.loginAccount match {
case Some(x) if(x.isAdmin) => action(repository)
case Some(x) if(paths(0) == x.userName) => action(repository)
- case Some(x) if(getCollaborators(paths(0), paths(1)).contains(x.userName)) => action(repository)
+ case Some(x) if(getGroupMembers(repository.owner).exists(_.userName == x.userName)) => action(repository)
+ case Some(x) if(getCollaboratorUserNames(paths(0), paths(1)).contains(x.userName)) => action(repository)
case _ => Unauthorized()
}
}
@@ -136,21 +115,46 @@
}
/**
- * Allows only signed in users which can access the repository.
+ * Allows only signed in users who have read permission for the repository.
*/
-trait ReadableUsersAuthenticator { self: ControllerBase with RepositoryService =>
+trait ReadableUsersAuthenticator { self: ControllerBase with RepositoryService with AccountService =>
protected def readableUsersOnly(action: (RepositoryInfo) => Any) = { authenticate(action) }
protected def readableUsersOnly[T](action: (T, RepositoryInfo) => Any) = (form: T) => { authenticate(action(form, _)) }
private def authenticate(action: (RepositoryInfo) => Any) = {
{
defining(request.paths){ paths =>
- getRepository(paths(0), paths(1), baseUrl).map { repository =>
+ getRepository(paths(0), paths(1)).map { repository =>
context.loginAccount match {
case Some(x) if(x.isAdmin) => action(repository)
case Some(x) if(!repository.repository.isPrivate) => action(repository)
case Some(x) if(paths(0) == x.userName) => action(repository)
- case Some(x) if(getCollaborators(paths(0), paths(1)).contains(x.userName)) => action(repository)
+ case Some(x) if(getGroupMembers(repository.owner).exists(_.userName == x.userName)) => action(repository)
+ case Some(x) if(getCollaboratorUserNames(paths(0), paths(1)).contains(x.userName)) => action(repository)
+ case _ => Unauthorized()
+ }
+ } getOrElse NotFound()
+ }
+ }
+ }
+}
+
+/**
+ * Allows only signed in users who have write permission for the repository.
+ */
+trait WritableUsersAuthenticator { self: ControllerBase with RepositoryService with AccountService =>
+ protected def writableUsersOnly(action: (RepositoryInfo) => Any) = { authenticate(action) }
+ protected def writableUsersOnly[T](action: (T, RepositoryInfo) => Any) = (form: T) => { authenticate(action(form, _)) }
+
+ private def authenticate(action: (RepositoryInfo) => Any) = {
+ {
+ defining(request.paths){ paths =>
+ getRepository(paths(0), paths(1)).map { repository =>
+ context.loginAccount match {
+ case Some(x) if(x.isAdmin) => action(repository)
+ case Some(x) if(paths(0) == x.userName) => action(repository)
+ case Some(x) if(getGroupMembers(repository.owner).exists(_.userName == x.userName)) => action(repository)
+ case Some(x) if(getCollaboratorUserNames(paths(0), paths(1), Seq(Role.ADMIN, Role.DEVELOPER)).contains(x.userName)) => action(repository)
case _ => Unauthorized()
}
} getOrElse NotFound()
diff --git a/src/main/scala/gitbucket/core/util/ControlUtil.scala b/src/main/scala/gitbucket/core/util/ControlUtil.scala
index 268e692..74569ac 100644
--- a/src/main/scala/gitbucket/core/util/ControlUtil.scala
+++ b/src/main/scala/gitbucket/core/util/ControlUtil.scala
@@ -1,8 +1,6 @@
package gitbucket.core.util
import org.eclipse.jgit.api.Git
-import org.eclipse.jgit.revwalk.RevWalk
-import org.eclipse.jgit.treewalk.TreeWalk
import scala.util.control.Exception._
import scala.language.reflectiveCalls
@@ -22,6 +20,20 @@
}
}
+ def using[A <% { def close(): Unit }, B <% { def close(): Unit }, C](resource1: A, resource2: B)(f: (A, B) => C): C =
+ try f(resource1, resource2) finally {
+ if(resource1 != null){
+ ignoring(classOf[Throwable]) {
+ resource1.close()
+ }
+ }
+ if(resource2 != null){
+ ignoring(classOf[Throwable]) {
+ resource2.close()
+ }
+ }
+ }
+
def using[T](git: Git)(f: Git => T): T =
try f(git) finally git.getRepository.close()
@@ -31,12 +43,6 @@
git2.getRepository.close()
}
- def using[T](revWalk: RevWalk)(f: RevWalk => T): T =
- try f(revWalk) finally revWalk.release()
-
- def using[T](treeWalk: TreeWalk)(f: TreeWalk => T): T =
- try f(treeWalk) finally treeWalk.release()
-
def ignore[T](f: => Unit): Unit = try {
f
} catch {
diff --git a/src/main/scala/gitbucket/core/util/DatabaseConfig.scala b/src/main/scala/gitbucket/core/util/DatabaseConfig.scala
index ffe7cee..fb8d184 100644
--- a/src/main/scala/gitbucket/core/util/DatabaseConfig.scala
+++ b/src/main/scala/gitbucket/core/util/DatabaseConfig.scala
@@ -1,19 +1,97 @@
package gitbucket.core.util
import com.typesafe.config.ConfigFactory
-import Directory.DatabaseHome
+import java.io.File
+import Directory._
+import liquibase.database.AbstractJdbcDatabase
+import liquibase.database.core.{PostgresDatabase, MySQLDatabase, H2Database}
+import org.apache.commons.io.FileUtils
object DatabaseConfig {
- private val config = ConfigFactory.load("database")
- private val dbUrl = config.getString("db.url")
+ private lazy val config = {
+ val file = new File(GitBucketHome, "database.conf")
+ if(!file.exists){
+ FileUtils.write(file,
+ """db {
+ | url = "jdbc:h2:${DatabaseHome};MVCC=true"
+ | user = "sa"
+ | password = "sa"
+ |# connectionTimeout = 30000
+ |# idleTimeout = 600000
+ |# maxLifetime = 1800000
+ |# minimumIdle = 10
+ |# maximumPoolSize = 10
+ |}
+ |""".stripMargin, "UTF-8")
+ }
+ ConfigFactory.parseFile(file)
+ }
+
+ private lazy val dbUrl = config.getString("db.url")
def url(directory: Option[String]): String =
dbUrl.replace("${DatabaseHome}", directory.getOrElse(DatabaseHome))
- val url: String = url(None)
- val user: String = config.getString("db.user")
- val password: String = config.getString("db.password")
- val driver: String = config.getString("db.driver")
+ lazy val url : String = url(None)
+ lazy val user : String = config.getString("db.user")
+ lazy val password : String = config.getString("db.password")
+ lazy val jdbcDriver : String = DatabaseType(url).jdbcDriver
+ lazy val slickDriver : slick.driver.JdbcProfile = DatabaseType(url).slickDriver
+ lazy val liquiDriver : AbstractJdbcDatabase = DatabaseType(url).liquiDriver
+ lazy val connectionTimeout : Option[Long] = getOptionValue("db.connectionTimeout", config.getLong)
+ lazy val idleTimeout : Option[Long] = getOptionValue("db.idleTimeout" , config.getLong)
+ lazy val maxLifetime : Option[Long] = getOptionValue("db.maxLifetime" , config.getLong)
+ lazy val minimumIdle : Option[Int] = getOptionValue("db.minimumIdle" , config.getInt)
+ lazy val maximumPoolSize : Option[Int] = getOptionValue("db.maximumPoolSize" , config.getInt)
+ private def getOptionValue[T](path: String, f: String => T): Option[T] = {
+ if(config.hasPath(path)) Some(f(path)) else None
+ }
+
+}
+
+sealed trait DatabaseType {
+ val jdbcDriver: String
+ val slickDriver: slick.driver.JdbcProfile
+ val liquiDriver: AbstractJdbcDatabase
+}
+
+object DatabaseType {
+
+ def apply(url: String): DatabaseType = {
+ if(url.startsWith("jdbc:h2:")){
+ H2
+ } else if(url.startsWith("jdbc:mysql:")){
+ MySQL
+ } else if(url.startsWith("jdbc:postgresql:")){
+ PostgreSQL
+ } else {
+ throw new IllegalArgumentException(s"${url} is not supported.")
+ }
+ }
+
+ object H2 extends DatabaseType {
+ val jdbcDriver = "org.h2.Driver"
+ val slickDriver = slick.driver.H2Driver
+ val liquiDriver = new H2Database()
+ }
+
+ object MySQL extends DatabaseType {
+ val jdbcDriver = "com.mysql.jdbc.Driver"
+ val slickDriver = slick.driver.MySQLDriver
+ val liquiDriver = new MySQLDatabase()
+ }
+
+ object PostgreSQL extends DatabaseType {
+ val jdbcDriver = "org.postgresql.Driver2"
+ val slickDriver = new slick.driver.PostgresDriver {
+ override def quoteIdentifier(id: String): String = {
+ val s = new StringBuilder(id.length + 4) append '"'
+ for(c <- id) if(c == '"') s append "\"\"" else s append c.toLower
+ (s append '"').toString
+ }
+ }
+ val liquiDriver = new PostgresDatabase()
+ }
}
diff --git a/src/main/scala/gitbucket/core/util/Directory.scala b/src/main/scala/gitbucket/core/util/Directory.scala
index de97bbb..e73bca8 100644
--- a/src/main/scala/gitbucket/core/util/Directory.scala
+++ b/src/main/scala/gitbucket/core/util/Directory.scala
@@ -1,11 +1,9 @@
package gitbucket.core.util
import java.io.File
-import ControlUtil._
-import org.apache.commons.io.FileUtils
/**
- * Provides directories used by GitBucket.
+ * Provides directory locations used by GitBucket.
*/
object Directory {
@@ -51,6 +49,12 @@
new File(s"${RepositoryHome}/${owner}/${repository}/comments")
/**
+ * Directory for files which are attached to issue.
+ */
+ def getLfsDir(owner: String, repository: String): File =
+ new File(s"${RepositoryHome}/${owner}/${repository}/lfs")
+
+ /**
* Directory for uploaded files by the specified user.
*/
def getUserUploadDir(userName: String): File = new File(s"${GitBucketHome}/data/${userName}/files")
@@ -73,12 +77,6 @@
def getPluginCacheDir(): File = new File(s"${TemporaryHome}/_plugins")
/**
- * Temporary directory which is used to create an archive to download repository contents.
- */
- def getDownloadWorkDir(owner: String, repository: String, sessionId: String): File =
- new File(getTemporaryDir(owner, repository), s"download/${sessionId}")
-
- /**
* Substance directory of the wiki repository.
*/
def getWikiRepositoryDir(owner: String, repository: String): File =
diff --git a/src/main/scala/gitbucket/core/util/FileUtil.scala b/src/main/scala/gitbucket/core/util/FileUtil.scala
index d3428fb..4836c23 100644
--- a/src/main/scala/gitbucket/core/util/FileUtil.scala
+++ b/src/main/scala/gitbucket/core/util/FileUtil.scala
@@ -1,7 +1,7 @@
package gitbucket.core.util
import org.apache.commons.io.FileUtils
-import java.net.URLConnection
+import org.apache.tika.Tika
import java.io.File
import ControlUtil._
import scala.util.Random
@@ -9,8 +9,8 @@
object FileUtil {
def getMimeType(name: String): String =
- defining(URLConnection.getFileNameMap()){ fileNameMap =>
- fileNameMap.getContentTypeFor(name) match {
+ defining(new Tika()){ tika =>
+ tika.detect(name) match {
case null => "application/octet-stream"
case mimeType => mimeType
}
@@ -28,6 +28,8 @@
def isImage(name: String): Boolean = getMimeType(name).startsWith("image/")
+ def isUploadableType(name: String): Boolean = mimeTypeWhiteList contains getMimeType(name)
+
def isLarge(size: Long): Boolean = (size > 1024 * 1000)
def isText(content: Array[Byte]): Boolean = !content.contains(0)
@@ -50,4 +52,18 @@
FileUtils.deleteDirectory(dir)
}
}
+
+ val mimeTypeWhiteList: Array[String] = Array(
+ "application/pdf",
+ "application/vnd.openxmlformats-officedocument.presentationml.presentation",
+ "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
+ "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
+ "image/gif",
+ "image/jpeg",
+ "image/png",
+ "text/plain")
+
+ def getLfsFilePath(owner: String, repository: String, oid: String): String =
+ Directory.getLfsDir(owner, repository) + "/" + oid
+
}
diff --git a/src/main/scala/gitbucket/core/util/Implicits.scala b/src/main/scala/gitbucket/core/util/Implicits.scala
index 13c316a..d93ebe2 100644
--- a/src/main/scala/gitbucket/core/util/Implicits.scala
+++ b/src/main/scala/gitbucket/core/util/Implicits.scala
@@ -4,6 +4,8 @@
import gitbucket.core.controller.Context
import gitbucket.core.servlet.Database
+import java.util.regex.Pattern.quote
+
import javax.servlet.http.{HttpSession, HttpServletRequest}
import scala.util.matching.Regex
@@ -65,6 +67,7 @@
def paths: Array[String] = (request.getRequestURI.substring(request.getContextPath.length + 1) match{
case path if path.startsWith("api/v3/repos/") => path.substring(13/* "/api/v3/repos".length */)
+ case path if path.startsWith("api/v3/orgs/") => path.substring(12/* "/api/v3/orgs".length */)
case path => path
}).split("/")
@@ -72,15 +75,16 @@
def hasAttribute(name: String): Boolean = request.getAttribute(name) != null
+ def gitRepositoryPath: String = request.getRequestURI.replaceFirst("^" + quote(request.getContextPath) + "/git/", "/")
+
+ def baseUrl:String = {
+ val url = request.getRequestURL.toString
+ val len = url.length - (request.getRequestURI.length - request.getContextPath.length)
+ url.substring(0, len).stripSuffix("/")
+ }
}
implicit class RichSession(session: HttpSession){
-
- def putAndGet[T](key: String, value: T): T = {
- session.setAttribute(key, value)
- value
- }
-
def getAndRemove[T](key: String): Option[T] = {
val value = session.getAttribute(key).asInstanceOf[T]
if(value == null){
diff --git a/src/main/scala/gitbucket/core/util/JDBCUtil.scala b/src/main/scala/gitbucket/core/util/JDBCUtil.scala
index 41a2b1a..edc41ec 100644
--- a/src/main/scala/gitbucket/core/util/JDBCUtil.scala
+++ b/src/main/scala/gitbucket/core/util/JDBCUtil.scala
@@ -1,12 +1,18 @@
package gitbucket.core.util
+import java.io._
import java.sql._
+import java.text.SimpleDateFormat
import ControlUtil._
+import scala.annotation.tailrec
import scala.collection.mutable.ListBuffer
/**
* Provides implicit class which extends java.sql.Connection.
- * This is used in automatic migration in [[servlet.AutoUpdateListener]].
+ * This is used in following points:
+ *
+ * - Automatic migration in [[gitbucket.core.servlet.InitializeListener]]
+ * - Data importing / exporting in [[gitbucket.core.controller.SystemSettingsController]] and [[gitbucket.core.controller.FileUploadController]]
*/
object JDBCUtil {
@@ -58,6 +64,176 @@
}
}
+ def importAsSQL(in: InputStream): Unit = {
+ conn.setAutoCommit(false)
+ try {
+ using(in){ in =>
+ var out = new ByteArrayOutputStream()
+
+ var length = 0
+ val bytes = new scala.Array[Byte](1024 * 8)
+ var stringLiteral = false
+
+ while({ length = in.read(bytes); length != -1 }){
+ for(i <- 0 to length - 1){
+ val c = bytes(i)
+ if(c == '\''){
+ stringLiteral = !stringLiteral
+ }
+ if(c == ';' && !stringLiteral){
+ val sql = new String(out.toByteArray, "UTF-8")
+ conn.update(sql.trim)
+ out = new ByteArrayOutputStream()
+ } else {
+ out.write(c)
+ }
+ }
+ }
+
+ val remain = out.toByteArray
+ if(remain.length != 0){
+ val sql = new String(remain, "UTF-8")
+ conn.update(sql.trim)
+ }
+ }
+ conn.commit()
+
+ } catch {
+ case e: Exception => {
+ conn.rollback()
+ throw e
+ }
+ }
+ }
+
+ def exportAsSQL(targetTables: Seq[String]): File = {
+ val dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss")
+ val file = File.createTempFile("gitbucket-export-", ".sql")
+
+ using(new FileOutputStream(file)) { out =>
+ val dbMeta = conn.getMetaData
+ val allTablesInDatabase = allTablesOrderByDependencies(dbMeta)
+
+ allTablesInDatabase.reverse.foreach { tableName =>
+ if (targetTables.contains(tableName)) {
+ out.write(s"DELETE FROM ${tableName};\n".getBytes("UTF-8"))
+ }
+ }
+
+ allTablesInDatabase.foreach { tableName =>
+ if (targetTables.contains(tableName)) {
+ val sb = new StringBuilder()
+ select(s"SELECT * FROM ${tableName}") { rs =>
+ sb.append(s"INSERT INTO ${tableName} (")
+
+ val rsMeta = rs.getMetaData
+ val columns = (1 to rsMeta.getColumnCount).map { i =>
+ (rsMeta.getColumnName(i), rsMeta.getColumnType(i))
+ }
+ sb.append(columns.map(_._1).mkString(", "))
+ sb.append(") VALUES (")
+
+ val values = columns.map { case (columnName, columnType) =>
+ if(rs.getObject(columnName) == null){
+ null
+ } else {
+ columnType match {
+ case Types.BOOLEAN | Types.BIT => rs.getBoolean(columnName)
+ case Types.VARCHAR | Types.CLOB | Types.CHAR | Types.LONGVARCHAR => rs.getString(columnName)
+ case Types.INTEGER => rs.getInt(columnName)
+ case Types.TIMESTAMP => rs.getTimestamp(columnName)
+ }
+ }
+ }
+
+ val columnValues = values.map { value =>
+ value match {
+ case x: String => "'" + x.replace("'", "''") + "'"
+ case x: Timestamp => "'" + dateFormat.format(x) + "'"
+ case null => "NULL"
+ case x => x
+ }
+ }
+ sb.append(columnValues.mkString(", "))
+ sb.append(");\n")
+ }
+
+ out.write(sb.toString.getBytes("UTF-8"))
+ }
+ }
+ }
+
+ file
+ }
+
+ def allTableNames(): Seq[String] = {
+ using(conn.getMetaData.getTables(null, null, "%", Seq("TABLE").toArray)) { rs =>
+ val tableNames = new ListBuffer[String]
+ while (rs.next) {
+ val name = rs.getString("TABLE_NAME").toUpperCase
+ if (name != "VERSIONS" && name != "PLUGIN") {
+ tableNames += name
+ }
+ }
+ tableNames.toSeq
+ }
+ }
+
+ private def childTables(meta: DatabaseMetaData, tableName: String): Seq[String] = {
+ val normalizedTableName =
+ if(meta.getDatabaseProductName == "PostgreSQL"){
+ tableName.toLowerCase
+ } else {
+ tableName
+ }
+
+ using(meta.getExportedKeys(null, null, normalizedTableName)) { rs =>
+ val children = new ListBuffer[String]
+ while (rs.next) {
+ val childTableName = rs.getString("FKTABLE_NAME").toUpperCase
+ if(!children.contains(childTableName)){
+ children += childTableName
+ children ++= childTables(meta, childTableName)
+ }
+ }
+ children.distinct.toSeq
+ }
+ }
+
+
+ private def allTablesOrderByDependencies(meta: DatabaseMetaData): Seq[String] = {
+ val tables = allTableNames.map { tableName =>
+ val result = TableDependency(tableName, childTables(meta, tableName))
+ result
+ }
+
+ val edges = tables.flatMap { table =>
+ table.children.map { child => (table.tableName, child) }
+ }
+
+ tsort(edges).toSeq
+ }
+
+ case class TableDependency(tableName: String, children: Seq[String])
+
+
+ def tsort[A](edges: Traversable[(A, A)]): Iterable[A] = {
+ @tailrec
+ def tsort(toPreds: Map[A, Set[A]], done: Iterable[A]): Iterable[A] = {
+ val (noPreds, hasPreds) = toPreds.partition { _._2.isEmpty }
+ if (noPreds.isEmpty) {
+ if (hasPreds.isEmpty) done else sys.error(hasPreds.toString)
+ } else {
+ val found = noPreds.map { _._1 }
+ tsort(hasPreds.mapValues { _ -- found }, done ++ found)
+ }
+ }
+
+ val toPred = edges.foldLeft(Map[A, Set[A]]()) { (acc, e) =>
+ acc + (e._1 -> acc.getOrElse(e._1, Set())) + (e._2 -> (acc.getOrElse(e._2, Set()) + e._1))
+ }
+ tsort(toPred, Seq())
+ }
}
}
diff --git a/src/main/scala/gitbucket/core/util/JGitUtil.scala b/src/main/scala/gitbucket/core/util/JGitUtil.scala
index 01e52c9..786edca 100644
--- a/src/main/scala/gitbucket/core/util/JGitUtil.scala
+++ b/src/main/scala/gitbucket/core/util/JGitUtil.scala
@@ -5,6 +5,7 @@
import Directory._
import StringUtil._
import ControlUtil._
+
import scala.annotation.tailrec
import scala.collection.JavaConverters._
import org.eclipse.jgit.lib._
@@ -16,7 +17,11 @@
import org.eclipse.jgit.errors.{ConfigInvalidException, MissingObjectException}
import org.eclipse.jgit.transport.RefSpec
import java.util.Date
-import org.eclipse.jgit.api.errors.{JGitInternalException, InvalidRefNameException, RefAlreadyExistsException, NoHeadException}
+import java.util.concurrent.TimeUnit
+import java.util.function.Consumer
+
+import org.cache2k.{Cache2kBuilder, CacheEntry}
+import org.eclipse.jgit.api.errors.{InvalidRefNameException, JGitInternalException, NoHeadException, RefAlreadyExistsException}
import org.eclipse.jgit.dircache.DirCacheEntry
import org.slf4j.LoggerFactory
@@ -32,15 +37,11 @@
*
* @param owner the user name of the repository owner
* @param name the repository name
- * @param url the repository URL
- * @param commitCount the commit count. If the repository has over 1000 commits then this property is 1001.
* @param branchList the list of branch names
* @param tags the list of tags
*/
- case class RepositoryInfo(owner: String, name: String, url: String, commitCount: Int, branchList: List[String], tags: List[TagInfo]){
- def this(owner: String, name: String, baseUrl: String) = {
- this(owner, name, s"${baseUrl}/git/${owner}/${name}.git", 0, Nil, Nil)
- }
+ case class RepositoryInfo(owner: String, name: String, branchList: List[String], tags: List[TagInfo]){
+ def this(owner: String, name: String) = this(owner, name, Nil, Nil)
}
/**
@@ -100,8 +101,20 @@
def isDifferentFromAuthor: Boolean = authorName != committerName || authorEmailAddress != committerEmailAddress
}
- case class DiffInfo(changeType: ChangeType, oldPath: String, newPath: String, oldContent: Option[String], newContent: Option[String],
- oldIsImage: Boolean, newIsImage: Boolean, oldObjectId: Option[String], newObjectId: Option[String])
+ case class DiffInfo(
+ changeType: ChangeType,
+ oldPath: String,
+ newPath: String,
+ oldContent: Option[String],
+ newContent: Option[String],
+ oldIsImage: Boolean,
+ newIsImage: Boolean,
+ oldObjectId: Option[String],
+ newObjectId: Option[String],
+ oldMode: String,
+ newMode: String,
+ tooLarge: Boolean
+ )
/**
* The file content data for the file content view of the repository viewer.
@@ -158,20 +171,54 @@
revWalk.dispose
revCommit
}
-
+
+ private val cache = new Cache2kBuilder[String, Int]() {}
+ .name("commit-count")
+ .expireAfterWrite(24, TimeUnit.HOURS)
+ .entryCapacity(10000)
+ .build()
+
+ def removeCache(git: Git): Unit = {
+ val dir = git.getRepository.getDirectory
+ val keyPrefix = dir.getAbsolutePath + "@"
+
+ cache.forEach(new Consumer[CacheEntry[String, Int]] {
+ override def accept(entry: CacheEntry[String, Int]): Unit = {
+ if(entry.getKey.startsWith(keyPrefix)){
+ cache.remove(entry.getKey)
+ }
+ }
+ })
+ }
+
+ /**
+ * Returns the number of commits in the specified branch or commit.
+ * If the specified branch has over 10000 commits, this method returns 100001.
+ */
+ def getCommitCount(owner: String, repository: String, branch: String): Int = {
+ val dir = getRepositoryDir(owner, repository)
+ val key = dir.getAbsolutePath + "@" + branch
+ val entry = cache.getEntry(key)
+
+ if(entry == null) {
+ using(Git.open(dir)) { git =>
+ val commitId = git.getRepository.resolve(branch)
+ val commitCount = git.log.add(commitId).call.iterator.asScala.take(10001).size
+ cache.put(key, commitCount)
+ commitCount
+ }
+ } else {
+ entry.getValue
+ }
+ }
+
/**
* Returns the repository information. It contains branch names and tag names.
*/
- def getRepositoryInfo(owner: String, repository: String, baseUrl: String): RepositoryInfo = {
+ def getRepositoryInfo(owner: String, repository: String): RepositoryInfo = {
using(Git.open(getRepositoryDir(owner, repository))){ git =>
try {
- // get commit count
- val commitCount = git.log.all.call.iterator.asScala.map(_ => 1).take(10001).sum
-
- RepositoryInfo(
- owner, repository, s"${baseUrl}/git/${owner}/${repository}.git",
- // commit count
- commitCount,
+ RepositoryInfo(owner, repository,
// branches
git.branchList.call.asScala.map { ref =>
ref.getName.stripPrefix("refs/heads/")
@@ -184,9 +231,7 @@
)
} catch {
// not initialized
- case e: NoHeadException => RepositoryInfo(
- owner, repository, s"${baseUrl}/git/${owner}/${repository}.git", 0, Nil, Nil)
-
+ case e: NoHeadException => RepositoryInfo(owner, repository, Nil, Nil)
}
}
}
@@ -201,8 +246,8 @@
*/
def getFileList(git: Git, revision: String, path: String = "."): List[FileInfo] = {
using(new RevWalk(git.getRepository)){ revWalk =>
- val objectId = git.getRepository.resolve(revision)
- if(objectId==null) return Nil
+ val objectId = git.getRepository.resolve(revision)
+ if(objectId == null) return Nil
val revCommit = revWalk.parseCommit(objectId)
def useTreeWalk(rev:RevCommit)(f:TreeWalk => Any): Unit = if (path == ".") {
@@ -244,14 +289,14 @@
revIterator:java.util.Iterator[RevCommit]): List[(ObjectId, FileMode, String, Option[String], RevCommit)] ={
if(restList.isEmpty){
result
- }else if(!revIterator.hasNext){ // maybe, revCommit has only 1 log. other case, restList be empty
- result ++ restList.map{ case (tuple, map) => tupleAdd(tuple, map.values.headOption.getOrElse(revCommit)) }
- }else{
+ } else if(!revIterator.hasNext){ // maybe, revCommit has only 1 log. other case, restList be empty
+ result ++ restList.map { case (tuple, map) => tupleAdd(tuple, map.values.headOption.getOrElse(revCommit)) }
+ } else {
val newCommit = revIterator.next
- val (thisTimeChecks,skips) = restList.partition{ case (tuple, parentsMap) => parentsMap.contains(newCommit) }
+ val (thisTimeChecks,skips) = restList.partition { case (tuple, parentsMap) => parentsMap.contains(newCommit) }
if(thisTimeChecks.isEmpty){
findLastCommits(result, restList, revIterator)
- }else{
+ } else {
var nextRest = skips
var nextResult = result
// Map[(name, oid), (tuple, parentsMap)]
@@ -259,20 +304,20 @@
lazy val newParentsMap = newCommit.getParents.map(_ -> newCommit).toMap
useTreeWalk(newCommit){ walk =>
while(walk.next){
- rest.remove(walk.getNameString -> walk.getObjectId(0)).map{ case (tuple, _) =>
+ rest.remove(walk.getNameString -> walk.getObjectId(0)).map { case (tuple, _) =>
if(newParentsMap.isEmpty){
nextResult +:= tupleAdd(tuple, newCommit)
- }else{
+ } else {
nextRest +:= tuple -> newParentsMap
}
}
}
}
- rest.values.map{ case (tuple, parentsMap) =>
+ rest.values.map { case (tuple, parentsMap) =>
val restParentsMap = parentsMap - newCommit
if(restParentsMap.isEmpty){
nextResult +:= tupleAdd(tuple, parentsMap(newCommit))
- }else{
+ } else {
nextRest +:= tuple -> restParentsMap
}
}
@@ -284,7 +329,7 @@
var fileList: List[(ObjectId, FileMode, String, Option[String])] = Nil
useTreeWalk(revCommit){ treeWalk =>
while (treeWalk.next()) {
- val linkUrl =if (treeWalk.getFileMode(0) == FileMode.GITLINK) {
+ val linkUrl = if (treeWalk.getFileMode(0) == FileMode.GITLINK) {
getSubmodules(git, revCommit.getTree).find(_.path == treeWalk.getPathString).map(_.url)
} else None
fileList +:= (treeWalk.getObjectId(0), treeWalk.getFileMode(0), treeWalk.getNameString, linkUrl)
@@ -334,7 +379,7 @@
def getTreeId(git: Git, revision: String): Option[String] = {
using(new RevWalk(git.getRepository)){ revWalk =>
val objectId = git.getRepository.resolve(revision)
- if(objectId==null) return None
+ if(objectId == null) return None
val revCommit = revWalk.parseCommit(objectId)
Some(revCommit.getTree.name)
}
@@ -346,7 +391,7 @@
def getAllFileListByTreeId(git: Git, treeId: String): List[String] = {
using(new RevWalk(git.getRepository)){ revWalk =>
val objectId = git.getRepository.resolve(treeId+"^{tree}")
- if(objectId==null) return Nil
+ if(objectId == null) return Nil
using(new TreeWalk(git.getRepository)){ treeWalk =>
treeWalk.addTree(objectId)
treeWalk.setRecursive(true)
@@ -495,11 +540,35 @@
while(treeWalk.next){
val newIsImage = FileUtil.isImage(treeWalk.getPathString)
buffer.append((if(!fetchContent){
- DiffInfo(ChangeType.ADD, null, treeWalk.getPathString, None, None, false, newIsImage, None, Option(treeWalk.getObjectId(0)).map(_.name))
+ DiffInfo(
+ changeType = ChangeType.ADD,
+ oldPath = null,
+ newPath = treeWalk.getPathString,
+ oldContent = None,
+ newContent = None,
+ oldIsImage = false,
+ newIsImage = newIsImage,
+ oldObjectId = None,
+ newObjectId = Option(treeWalk.getObjectId(0)).map(_.name),
+ oldMode = treeWalk.getFileMode(0).toString,
+ newMode = treeWalk.getFileMode(0).toString,
+ tooLarge = false
+ )
} else {
- DiffInfo(ChangeType.ADD, null, treeWalk.getPathString, None,
- JGitUtil.getContentFromId(git, treeWalk.getObjectId(0), false).filter(FileUtil.isText).map(convertFromByteArray),
- false, newIsImage, None, Option(treeWalk.getObjectId(0)).map(_.name))
+ DiffInfo(
+ changeType = ChangeType.ADD,
+ oldPath = null,
+ newPath = treeWalk.getPathString,
+ oldContent = None,
+ newContent = JGitUtil.getContentFromId(git, treeWalk.getObjectId(0), false).filter(FileUtil.isText).map(convertFromByteArray),
+ oldIsImage = false,
+ newIsImage = newIsImage,
+ oldObjectId = None,
+ newObjectId = Option(treeWalk.getObjectId(0)).map(_.name),
+ oldMode = treeWalk.getFileMode(0).toString,
+ newMode = treeWalk.getFileMode(0).toString,
+ tooLarge = false
+ )
}))
}
(buffer.toList, None)
@@ -518,16 +587,58 @@
import scala.collection.JavaConverters._
git.getRepository.getConfig.setString("diff", null, "renames", "copies")
- git.diff.setNewTree(newTreeIter).setOldTree(oldTreeIter).call.asScala.map { diff =>
- val oldIsImage = FileUtil.isImage(diff.getOldPath)
- val newIsImage = FileUtil.isImage(diff.getNewPath)
- if(!fetchContent || oldIsImage || newIsImage){
- DiffInfo(diff.getChangeType, diff.getOldPath, diff.getNewPath, None, None, oldIsImage, newIsImage, Option(diff.getOldId).map(_.name), Option(diff.getNewId).map(_.name))
+
+ val diffs = git.diff.setNewTree(newTreeIter).setOldTree(oldTreeIter).call.asScala
+ diffs.map { diff =>
+ if(diffs.size > 100){
+ DiffInfo(
+ changeType = diff.getChangeType,
+ oldPath = diff.getOldPath,
+ newPath = diff.getNewPath,
+ oldContent = None,
+ newContent = None,
+ oldIsImage = false,
+ newIsImage = false,
+ oldObjectId = Option(diff.getOldId).map(_.name),
+ newObjectId = Option(diff.getNewId).map(_.name),
+ oldMode = diff.getOldMode.toString,
+ newMode = diff.getNewMode.toString,
+ tooLarge = true
+ )
} else {
- DiffInfo(diff.getChangeType, diff.getOldPath, diff.getNewPath,
- JGitUtil.getContentFromId(git, diff.getOldId.toObjectId, false).filter(FileUtil.isText).map(convertFromByteArray),
- JGitUtil.getContentFromId(git, diff.getNewId.toObjectId, false).filter(FileUtil.isText).map(convertFromByteArray),
- oldIsImage, newIsImage, Option(diff.getOldId).map(_.name), Option(diff.getNewId).map(_.name))
+ val oldIsImage = FileUtil.isImage(diff.getOldPath)
+ val newIsImage = FileUtil.isImage(diff.getNewPath)
+ if(!fetchContent || oldIsImage || newIsImage){
+ DiffInfo(
+ changeType = diff.getChangeType,
+ oldPath = diff.getOldPath,
+ newPath = diff.getNewPath,
+ oldContent = None,
+ newContent = None,
+ oldIsImage = oldIsImage,
+ newIsImage = newIsImage,
+ oldObjectId = Option(diff.getOldId).map(_.name),
+ newObjectId = Option(diff.getNewId).map(_.name),
+ oldMode = diff.getOldMode.toString,
+ newMode = diff.getNewMode.toString,
+ tooLarge = false
+ )
+ } else {
+ DiffInfo(
+ changeType = diff.getChangeType,
+ oldPath = diff.getOldPath,
+ newPath = diff.getNewPath,
+ oldContent = JGitUtil.getContentFromId(git, diff.getOldId.toObjectId, false).filter(FileUtil.isText).map(convertFromByteArray),
+ newContent = JGitUtil.getContentFromId(git, diff.getNewId.toObjectId, false).filter(FileUtil.isText).map(convertFromByteArray),
+ oldIsImage = oldIsImage,
+ newIsImage = newIsImage,
+ oldObjectId = Option(diff.getOldId).map(_.name),
+ newObjectId = Option(diff.getNewId).map(_.name),
+ oldMode = diff.getOldMode.toString,
+ newMode = diff.getNewMode.toString,
+ tooLarge = false
+ )
+ }
}
}.toList
}
@@ -563,7 +674,7 @@
def initRepository(dir: java.io.File): Unit =
using(new RepositoryBuilder().setGitDir(dir).setBare.build){ repository =>
- repository.create
+ repository.create(true)
setReceivePack(repository)
}
@@ -622,12 +733,14 @@
val newHeadId = inserter.insert(newCommit)
inserter.flush()
- inserter.release()
+ inserter.close()
val refUpdate = git.getRepository.updateRef(ref)
refUpdate.setNewObjectId(newHeadId)
refUpdate.update()
+ removeCache(git)
+
newHeadId
}
@@ -713,7 +826,7 @@
def getContentFromId(git: Git, id: ObjectId, fetchLargeFile: Boolean): Option[Array[Byte]] = try {
using(git.getRepository.getObjectDatabase){ db =>
val loader = db.open(id)
- if(fetchLargeFile == false && FileUtil.isLarge(loader.getSize)){
+ if(loader.isLarge || (fetchLargeFile == false && FileUtil.isLarge(loader.getSize))){
None
} else {
Some(loader.getBytes)
@@ -724,6 +837,22 @@
}
/**
+ * Get objectLoader of the given object id from the Git repository.
+ *
+ * @param git the Git object
+ * @param id the object id
+ * @param f the function process ObjectLoader
+ * @return None if object does not exist
+ */
+ def getObjectLoaderFromId[A](git: Git, id: ObjectId)(f: ObjectLoader => A):Option[A] = try {
+ using(git.getRepository.getObjectDatabase){ db =>
+ Some(f(db.open(id)))
+ }
+ } catch {
+ case e: MissingObjectException => None
+ }
+
+ /**
* Returns all commit id in the specified repository.
*/
def getAllCommitIds(git: Git): Seq[String] = if(isEmpty(git)) {
@@ -737,14 +866,16 @@
existIds.toSeq
}
- def processTree(git: Git, id: ObjectId)(f: (String, CanonicalTreeParser) => Unit) = {
+ def processTree[T](git: Git, id: ObjectId)(f: (String, CanonicalTreeParser) => T): Seq[T] = {
using(new RevWalk(git.getRepository)){ revWalk =>
using(new TreeWalk(git.getRepository)){ treeWalk =>
val index = treeWalk.addTree(revWalk.parseTree(id))
treeWalk.setRecursive(true)
+ val result = new collection.mutable.ListBuffer[T]()
while(treeWalk.next){
- f(treeWalk.getPathString, treeWalk.getTree(index, classOf[CanonicalTreeParser]))
+ result += f(treeWalk.getPathString, treeWalk.getTree(index, classOf[CanonicalTreeParser]))
}
+ result.toSeq
}
}
}
@@ -782,6 +913,7 @@
/**
* Returns the last modified commit of specified path
+ *
* @param git the Git object
* @param startCommit the search base commit id
* @param path the path of target file or directory
@@ -791,7 +923,7 @@
return git.log.add(startCommit).addPath(path).setMaxCount(1).call.iterator.next
}
- def getBranches(owner: String, name: String, defaultBranch: String): Seq[BranchInfo] = {
+ def getBranches(owner: String, name: String, defaultBranch: String, origin: Boolean): Seq[BranchInfo] = {
using(Git.open(getRepositoryDir(owner, name))){ git =>
val repo = git.getRepository
val defaultObject = if (repo.getAllRefs.keySet().contains(defaultBranch)) {
@@ -802,20 +934,16 @@
git.branchList.call.asScala.map { ref =>
val walk = new RevWalk(repo)
- try{
- val defaultCommit = walk.parseCommit(defaultObject)
- val branchName = ref.getName.stripPrefix("refs/heads/")
- val branchCommit = if(branchName == defaultBranch){
- defaultCommit
- }else{
- walk.parseCommit(ref.getObjectId)
- }
- val when = branchCommit.getCommitterIdent.getWhen
- val committer = branchCommit.getCommitterIdent.getName
+ try {
+ val defaultCommit = walk.parseCommit(defaultObject)
+ val branchName = ref.getName.stripPrefix("refs/heads/")
+ val branchCommit = walk.parseCommit(ref.getObjectId)
+ val when = branchCommit.getCommitterIdent.getWhen
+ val committer = branchCommit.getCommitterIdent.getName
val committerEmail = branchCommit.getCommitterIdent.getEmailAddress
- val mergeInfo = if(branchName==defaultBranch){
+ val mergeInfo = if(origin && branchName == defaultBranch){
None
- }else{
+ } else {
walk.reset()
walk.setRevFilter( RevFilter.MERGE_BASE )
walk.markStart(branchCommit)
@@ -868,6 +996,7 @@
/**
* Returns sha1
+ *
* @param owner repository owner
* @param name repository name
* @param revstr A git object references expression
diff --git a/src/main/scala/gitbucket/core/util/Notifier.scala b/src/main/scala/gitbucket/core/util/Notifier.scala
index 54b01e3..1897598 100644
--- a/src/main/scala/gitbucket/core/util/Notifier.scala
+++ b/src/main/scala/gitbucket/core/util/Notifier.scala
@@ -1,7 +1,7 @@
package gitbucket.core.util
-import gitbucket.core.model.{Session, Issue}
-import gitbucket.core.service.{RepositoryService, AccountService, IssuesService, SystemSettingsService}
+import gitbucket.core.model.{Account, Issue, Session}
+import gitbucket.core.service.{AccountService, IssuesService, RepositoryService, SystemSettingsService}
import gitbucket.core.servlet.Database
import gitbucket.core.view.Markdown
@@ -9,35 +9,41 @@
import ExecutionContext.Implicits.global
import org.apache.commons.mail.{DefaultAuthenticator, HtmlEmail}
import org.slf4j.LoggerFactory
-
import gitbucket.core.controller.Context
import SystemSettingsService.Smtp
import ControlUtil.defining
trait Notifier extends RepositoryService with AccountService with IssuesService {
+
def toNotify(r: RepositoryService.RepositoryInfo, issue: Issue, content: String)
(msg: String => String)(implicit context: Context): Unit
- protected def recipients(issue: Issue)(notify: String => Unit)(implicit session: Session, context: Context) =
+ protected def recipients(issue: Issue, loginAccount: Account)(notify: String => Unit)(implicit session: Session) =
(
// individual repository's owner
issue.userName ::
+ // group members of group repository
+ getGroupMembers(issue.userName).map(_.userName) :::
// collaborators
- getCollaborators(issue.userName, issue.repositoryName) :::
+ getCollaboratorUserNames(issue.userName, issue.repositoryName) :::
// participants
issue.openedUserName ::
getComments(issue.userName, issue.repositoryName, issue.issueId).map(_.commentedUserName)
)
.distinct
- .withFilter ( _ != context.loginAccount.get.userName ) // the operation in person is excluded
- .foreach ( getAccountByUserName(_) filterNot (_.isGroupAccount) filterNot (LDAPUtil.isDummyMailAddress(_)) foreach (x => notify(x.mailAddress)) )
-
+ .withFilter ( _ != loginAccount.userName ) // the operation in person is excluded
+ .foreach (
+ getAccountByUserName(_)
+ .filterNot (_.isGroupAccount)
+ .filterNot (LDAPUtil.isDummyMailAddress(_))
+ .foreach (x => notify(x.mailAddress))
+ )
}
object Notifier {
// TODO We want to be able to switch to mock.
def apply(): Notifier = new SystemSettingsService {}.loadSystemSettings match {
- case settings if settings.notification => new Mailer(settings.smtp.get)
+ case settings if (settings.notification && settings.useSMTP) => new Mailer(settings.smtp.get)
case _ => new MockMailer
}
@@ -68,47 +74,67 @@
private val logger = LoggerFactory.getLogger(classOf[Mailer])
def toNotify(r: RepositoryService.RepositoryInfo, issue: Issue, content: String)
- (msg: String => String)(implicit context: Context) = {
- val database = Database()
+ (msg: String => String)(implicit context: Context): Unit = {
+ context.loginAccount.foreach { loginAccount =>
+ val database = Database()
- val f = Future {
- database withSession { implicit session =>
- defining(
- s"[${r.name}] ${issue.title} (#${issue.issueId})" ->
- msg(Markdown.toHtml(content, r, false, true, false))) { case (subject, msg) =>
- recipients(issue) { to =>
- val email = new HtmlEmail
- email.setHostName(smtp.host)
- email.setSmtpPort(smtp.port.get)
- smtp.user.foreach { user =>
- email.setAuthenticator(new DefaultAuthenticator(user, smtp.password.getOrElse("")))
- }
- smtp.ssl.foreach { ssl =>
- email.setSSLOnConnect(ssl)
- }
- smtp.fromAddress
- .map (_ -> smtp.fromName.orNull)
- .orElse (Some("notifications@gitbucket.com" -> context.loginAccount.get.userName))
- .foreach { case (address, name) =>
- email.setFrom(address, name)
- }
- email.setCharset("UTF-8")
- email.setSubject(subject)
- email.setHtmlMsg(msg)
-
- email.addTo(to).send
- }
+ val f = Future {
+ database withSession { implicit session =>
+ defining(
+ s"[${r.name}] ${issue.title} (#${issue.issueId})" ->
+ msg(Markdown.toHtml(
+ markdown = content,
+ repository = r,
+ enableWikiLink = false,
+ enableRefsLink = true,
+ enableAnchor = false,
+ enableLineBreaks = false
+ ))
+ ) { case (subject, msg) =>
+ recipients(issue, loginAccount) { to => send(to, subject, msg, loginAccount) }
+ }
}
+ "Notifications Successful."
}
- "Notifications Successful."
- }
- f onSuccess {
- case s => logger.debug(s)
- }
- f onFailure {
- case t => logger.error("Notifications Failed.", t)
+ f onSuccess {
+ case s => logger.debug(s)
+ }
+ f onFailure {
+ case t => logger.error("Notifications Failed.", t)
+ }
}
}
+
+ def send(to: String, subject: String, msg: String, loginAccount: Account): Unit = {
+ val email = new HtmlEmail
+ email.setHostName(smtp.host)
+ email.setSmtpPort(smtp.port.get)
+ smtp.user.foreach { user =>
+ email.setAuthenticator(new DefaultAuthenticator(user, smtp.password.getOrElse("")))
+ }
+ smtp.ssl.foreach { ssl =>
+ email.setSSLOnConnect(ssl)
+ if(ssl == true) {
+ email.setSslSmtpPort(smtp.port.get.toString)
+ }
+ }
+ smtp.starttls.foreach { starttls =>
+ email.setStartTLSEnabled(starttls)
+ email.setStartTLSRequired(starttls)
+ }
+ smtp.fromAddress
+ .map (_ -> smtp.fromName.getOrElse(loginAccount.userName))
+ .orElse (Some("notifications@gitbucket.com" -> loginAccount.userName))
+ .foreach { case (address, name) =>
+ email.setFrom(address, name)
+ }
+ email.setCharset("UTF-8")
+ email.setSubject(subject)
+ email.setHtmlMsg(msg)
+
+ email.addTo(to).send
+ }
+
}
class MockMailer extends Notifier {
def toNotify(r: RepositoryService.RepositoryInfo, issue: Issue, content: String)
diff --git a/src/main/scala/gitbucket/core/util/RepoitoryName.scala b/src/main/scala/gitbucket/core/util/RepoitoryName.scala
deleted file mode 100644
index 415b20b..0000000
--- a/src/main/scala/gitbucket/core/util/RepoitoryName.scala
+++ /dev/null
@@ -1,18 +0,0 @@
-package gitbucket.core.util
-
-case class RepositoryName(owner:String, name:String){
- val fullName = s"${owner}/${name}"
-}
-
-object RepositoryName{
- def apply(fullName: String): RepositoryName = {
- fullName.split("/").toList match {
- case owner :: name :: Nil => RepositoryName(owner, name)
- case _ => throw new IllegalArgumentException(s"${fullName} is not repositoryName (only 'owner/name')")
- }
- }
- def apply(repository: gitbucket.core.model.Repository): RepositoryName = RepositoryName(repository.userName, repository.repositoryName)
- def apply(repository: gitbucket.core.util.JGitUtil.RepositoryInfo): RepositoryName = RepositoryName(repository.owner, repository.name)
- def apply(repository: gitbucket.core.service.RepositoryService.RepositoryInfo): RepositoryName = RepositoryName(repository.owner, repository.name)
- def apply(repository: gitbucket.core.model.CommitStatus): RepositoryName = RepositoryName(repository.userName, repository.repositoryName)
-}
diff --git a/src/main/scala/gitbucket/core/util/RepositoryName.scala b/src/main/scala/gitbucket/core/util/RepositoryName.scala
new file mode 100644
index 0000000..e7d293d
--- /dev/null
+++ b/src/main/scala/gitbucket/core/util/RepositoryName.scala
@@ -0,0 +1,19 @@
+package gitbucket.core.util
+
+// TODO Move to gitbucket.core.api package?
+case class RepositoryName(owner:String, name:String){
+ val fullName = s"${owner}/${name}"
+}
+
+object RepositoryName{
+ def apply(fullName: String): RepositoryName = {
+ fullName.split("/").toList match {
+ case owner :: name :: Nil => RepositoryName(owner, name)
+ case _ => throw new IllegalArgumentException(s"${fullName} is not repositoryName (only 'owner/name')")
+ }
+ }
+ def apply(repository: gitbucket.core.model.Repository): RepositoryName = RepositoryName(repository.userName, repository.repositoryName)
+ def apply(repository: gitbucket.core.util.JGitUtil.RepositoryInfo): RepositoryName = RepositoryName(repository.owner, repository.name)
+ def apply(repository: gitbucket.core.service.RepositoryService.RepositoryInfo): RepositoryName = RepositoryName(repository.owner, repository.name)
+ def apply(repository: gitbucket.core.model.CommitStatus): RepositoryName = RepositoryName(repository.userName, repository.repositoryName)
+}
diff --git a/src/main/scala/gitbucket/core/util/StringUtil.scala b/src/main/scala/gitbucket/core/util/StringUtil.scala
index 5633598..0db3efe 100644
--- a/src/main/scala/gitbucket/core/util/StringUtil.scala
+++ b/src/main/scala/gitbucket/core/util/StringUtil.scala
@@ -1,13 +1,23 @@
package gitbucket.core.util
import java.net.{URLDecoder, URLEncoder}
+
import org.mozilla.universalchardet.UniversalDetector
import ControlUtil._
import org.apache.commons.io.input.BOMInputStream
import org.apache.commons.io.IOUtils
+import org.apache.commons.codec.binary.Base64
+
+import scala.util.control.Exception._
object StringUtil {
+ private lazy val BlowfishKey = {
+ // last 4 numbers in current timestamp
+ val time = System.currentTimeMillis.toString
+ time.substring(time.length - 4)
+ }
+
def sha1(value: String): String =
defining(java.security.MessageDigest.getInstance("SHA-1")){ md =>
md.update(value.getBytes)
@@ -20,12 +30,28 @@
md.digest.map(b => "%02x".format(b)).mkString
}
- def urlEncode(value: String): String = URLEncoder.encode(value, "UTF-8")
+ def encodeBlowfish(value: String): String = {
+ val spec = new javax.crypto.spec.SecretKeySpec(BlowfishKey.getBytes(), "Blowfish")
+ val cipher = javax.crypto.Cipher.getInstance("Blowfish")
+ cipher.init(javax.crypto.Cipher.ENCRYPT_MODE, spec)
+ new String(Base64.encodeBase64(cipher.doFinal(value.getBytes("UTF-8"))), "UTF-8")
+ }
+
+ def decodeBlowfish(value: String): String = {
+ val spec = new javax.crypto.spec.SecretKeySpec(BlowfishKey.getBytes(), "Blowfish")
+ val cipher = javax.crypto.Cipher.getInstance("Blowfish")
+ cipher.init(javax.crypto.Cipher.DECRYPT_MODE, spec)
+ new String(cipher.doFinal(Base64.decodeBase64(value)), "UTF-8")
+ }
+
+ def urlEncode(value: String): String = URLEncoder.encode(value, "UTF-8").replace("+", "%20")
def urlDecode(value: String): String = URLDecoder.decode(value, "UTF-8")
def splitWords(value: String): Array[String] = value.split("[ \\t ]+")
+ def isInteger(value: String): Boolean = allCatch opt { value.toInt } map(_ => true) getOrElse(false)
+
def escapeHtml(value: String): String =
value.replace("&", "&").replace("<", "<").replace(">", ">").replace("\"", """)
@@ -83,8 +109,9 @@
*@param message the message which may contains issue id
* @return the iterator of issue id
*/
- def extractIssueId(message: String): Iterator[String] =
- "(^|\\W)#(\\d+)(\\W|$)".r.findAllIn(message).matchData.map(_.group(2))
+ def extractIssueId(message: String): Seq[String] =
+ "(^|\\W)#(\\d+)(\\W|$)".r
+ .findAllIn(message).matchData.map(_.group(2)).toSeq.distinct
/**
* Extract close issue id like ```close #issueId ``` from the given message.
@@ -92,7 +119,8 @@
* @param message the message which may contains close command
* @return the iterator of issue id
*/
- def extractCloseId(message: String): Iterator[String] =
- "(?i)(?
- if(in != null){
- val sql = IOUtils.toString(in, "UTF-8")
- using(conn.createStatement()){ stmt =>
- logger.debug(sqlPath + "=" + sql)
- stmt.executeUpdate(sql)
- }
- }
- }
- }
-
-
- /**
- * MAJOR.MINOR
- */
- val versionString = s"${majorVersion}.${minorVersion}"
-
-}
-
-object Versions {
-
- private val logger = LoggerFactory.getLogger(Versions.getClass)
-
- def update(conn: Connection, headVersion: Version, currentVersion: Version, versions: Seq[Version], cl: ClassLoader)
- (save: Connection => Unit): Unit = {
- logger.debug("Start schema update")
- try {
- if(currentVersion == headVersion){
- logger.debug("No update")
- } else if(currentVersion.versionString != "0.0" && !versions.contains(currentVersion)){
- logger.warn(s"Skip migration because ${currentVersion.versionString} is illegal version.")
- } else {
- versions.takeWhile(_ != currentVersion).reverse.foreach(_.update(conn, cl))
- save(conn)
- logger.debug(s"Updated from ${currentVersion.versionString} to ${headVersion.versionString}")
- }
- } catch {
- case ex: Throwable => {
- logger.error("Failed to schema update", ex)
- ex.printStackTrace()
- conn.rollback()
- }
- }
- logger.debug("End schema update")
- }
-
-}
-
diff --git a/src/main/scala/gitbucket/core/view/LinkConverter.scala b/src/main/scala/gitbucket/core/view/LinkConverter.scala
index 59bebe6..00d4e88 100644
--- a/src/main/scala/gitbucket/core/view/LinkConverter.scala
+++ b/src/main/scala/gitbucket/core/view/LinkConverter.scala
@@ -7,31 +7,94 @@
trait LinkConverter { self: RequestCache =>
/**
- * Converts issue id, username and commit id to link.
+ * Creates a link to the issue or the pull request from the issue id.
*/
- protected def convertRefsLinks(value: String, repository: RepositoryService.RepositoryInfo,
- issueIdPrefix: String = "#", escapeHtml: Boolean = true)(implicit context: Context): String = {
+ protected def createIssueLink(repository: RepositoryService.RepositoryInfo, issueId: Int)(implicit context: Context): String = {
+ val userName = repository.repository.userName
+ val repositoryName = repository.repository.repositoryName
+
+ getIssue(userName, repositoryName, issueId.toString) match {
+ case Some(issue) if (issue.isPullRequest) =>
+ s"""Pull #${issueId}"""
+ case Some(_) =>
+ s"""Issue #${issueId}"""
+ case None =>
+ s"Unknown #${issueId}"
+ }
+ }
+
+
+ /**
+ * Converts issue id, username and commit id to link in the given text.
+ */
+ protected def convertRefsLinks(text: String, repository: RepositoryService.RepositoryInfo,
+ issueIdPrefix: String = "#", escapeHtml: Boolean = true)(implicit context: Context): String = {
// escape HTML tags
- val escaped = if(escapeHtml) value.replace("&", "&").replace("<", "<").replace(">", ">").replace("\"", """) else value
+ val escaped = if(escapeHtml) text.replace("&", "&").replace("<", "<").replace(">", ">").replace("\"", """) else text
escaped
- // convert issue id to link
- .replaceBy(("(?<=(^|\\W))" + issueIdPrefix + "([0-9]+)(?=(\\W|$))").r){ m =>
- getIssue(repository.owner, repository.name, m.group(2)) match {
- case Some(issue) if(issue.isPullRequest)
- => Some(s"""#${m.group(2)}""")
- case Some(_) => Some(s"""#${m.group(2)}""")
- case None => Some(s"""#${m.group(2)}""")
+ // convert username/project@SHA to link
+ .replaceBy("(?<=(^|\\W))([a-zA-Z0-9\\-_]+)/([a-zA-Z0-9\\-_\\.]+)@([a-f0-9]{40})(?=(\\W|$))".r){ m =>
+ getAccountByUserName(m.group(2)).map { _ =>
+ s"""${m.group(2)}/${m.group(3)}@${m.group(4).substring(0, 7)}"""
}
}
+
+ // convert username/project#Num to link
+ .replaceBy( ("(?<=(^|\\W))([a-zA-Z0-9\\-_]+)/([a-zA-Z0-9\\-_\\.]+)" + issueIdPrefix + "([0-9]+)(?=(\\W|$))").r){ m =>
+ getIssue(m.group(2), m.group(3), m.group(4)) match {
+ case Some(issue) if (issue.isPullRequest) =>
+ Some(s"""${m.group(2)}/${m.group(3)}#${m.group(4)}""")
+ case Some(_) =>
+ Some(s"""${m.group(2)}/${m.group(3)}#${m.group(4)}""")
+ case None =>
+ Some(s"""${m.group(2)}/${m.group(3)}#${m.group(4)}""")
+ }
+ }
+
+ // convert username@SHA to link
+ .replaceBy( ("(?<=(^|\\W))([a-zA-Z0-9\\-_]+)@([a-f0-9]{40})(?=(\\W|$))").r ) { m =>
+ getAccountByUserName(m.group(2)).map { _ =>
+ s"""${m.group(2)}@${m.group(3).substring(0, 7)}"""
+ }
+ }
+
+ // convert username#Num to link
+ .replaceBy( ("(?<=(^|\\W))([a-zA-Z0-9\\-_]+)" + issueIdPrefix + "([0-9]+)(?=(\\W|$))").r ) { m =>
+ getIssue(m.group(2), repository.name, m.group(3)) match {
+ case Some(issue) if(issue.isPullRequest) =>
+ Some(s"""${m.group(2)}#${m.group(3)}""")
+ case Some(_) =>
+ Some(s"""${m.group(2)}#${m.group(3)}""")
+ case None =>
+ Some(s"""${m.group(2)}#${m.group(3)}""")
+ }
+ }
+
+ // convert issue id to link
+ .replaceBy(("(?<=(^|\\W))(GH-|(?
+ val prefix = if(m.group(2) == "issue:") "#" else m.group(2)
+ getIssue(repository.owner, repository.name, m.group(3)) match {
+ case Some(issue) if(issue.isPullRequest) =>
+ Some(s"""${prefix}${m.group(3)}""")
+ case Some(_) =>
+ Some(s"""${prefix}${m.group(3)}""")
+ case None =>
+ Some(s"""${m.group(2)}${m.group(3)}""")
+ }
+ }
+
// convert @username to link
- .replaceBy("(?<=(^|\\W))@([a-zA-Z0-9\\-_]+)(?=(\\W|$))".r){ m =>
+ .replaceBy("(?<=(^|\\W))@([a-zA-Z0-9\\-_\\.]+)(?=(\\W|$))".r){ m =>
getAccountByUserName(m.group(2)).map { _ =>
s"""@${m.group(2)}"""
}
}
+
// convert commit id to link
- .replaceAll("(?<=(^|\\W))([a-f0-9]{40})(?=(\\W|$))", s"""$$2""")
+ .replaceBy("(?<=(^|[^\\w/@]))([a-f0-9]{40})(?=(\\W|$))".r){ m =>
+ Some(s"""${m.group(2).substring(0, 7)}""")
+ }
}
}
diff --git a/src/main/scala/gitbucket/core/view/Markdown.scala b/src/main/scala/gitbucket/core/view/Markdown.scala
index 859d29c..46b39b2 100644
--- a/src/main/scala/gitbucket/core/view/Markdown.scala
+++ b/src/main/scala/gitbucket/core/view/Markdown.scala
@@ -1,18 +1,14 @@
package gitbucket.core.view
import java.text.Normalizer
-import java.util.Locale
import java.util.regex.Pattern
+import java.util.Locale
import gitbucket.core.controller.Context
-import gitbucket.core.service.{RepositoryService, RequestCache, WikiService}
+import gitbucket.core.service.{RepositoryService, RequestCache}
import gitbucket.core.util.StringUtil
-import org.parboiled.common.StringUtils
-import org.pegdown.LinkRenderer.Rendering
-import org.pegdown._
-import org.pegdown.ast._
-
-import scala.collection.JavaConverters._
+import io.github.gitbucket.markedj._
+import io.github.gitbucket.markedj.Utils._
object Markdown {
@@ -23,8 +19,9 @@
* @param enableWikiLink if true then wiki style link is available in markdown
* @param enableRefsLink if true then issue reference (e.g. #123) is rendered as link
* @param enableAnchor if true then anchor for headline is generated
+ * @param enableLineBreaks if true then render line breaks as <br>
* @param enableTaskList if true then task list syntax is available
- * @param hasWritePermission
+ * @param hasWritePermission true if user has writable to ths given repository
* @param pages the list of existing Wiki pages
*/
def toHtml(markdown: String,
@@ -32,242 +29,164 @@
enableWikiLink: Boolean,
enableRefsLink: Boolean,
enableAnchor: Boolean,
+ enableLineBreaks: Boolean,
enableTaskList: Boolean = false,
hasWritePermission: Boolean = false,
pages: List[String] = Nil)(implicit context: Context): String = {
- // escape issue id
- val s = if(enableRefsLink){
- markdown.replaceAll("(?<=(\\W|^))#(\\d+)(?=(\\W|$))", "issue:$2")
- } else markdown
-
// escape task list
- val source = if(enableTaskList){
- GitBucketHtmlSerializer.escapeTaskList(s)
- } else s
+ val source = if(enableTaskList) escapeTaskList(markdown) else markdown
- val rootNode = new PegDownProcessor(
- Extensions.AUTOLINKS | Extensions.WIKILINKS | Extensions.FENCED_CODE_BLOCKS | Extensions.TABLES | Extensions.HARDWRAPS | Extensions.SUPPRESS_ALL_HTML
- ).parseMarkdown(source.toCharArray)
+ val options = new Options()
+ options.setSanitize(true)
+ options.setBreaks(enableLineBreaks)
- new GitBucketHtmlSerializer(markdown, repository, enableWikiLink, enableRefsLink, enableAnchor, enableTaskList, hasWritePermission, pages).toHtml(rootNode)
+ val renderer = new GitBucketMarkedRenderer(options, repository,
+ enableWikiLink, enableRefsLink, enableAnchor, enableTaskList, hasWritePermission, pages)
+
+ //helpers.decorateHtml(Marked.marked(source, options, renderer), repository)
+ Marked.marked(source, options, renderer)
}
-}
-class GitBucketLinkRender(
- context: Context,
- repository: RepositoryService.RepositoryInfo,
- enableWikiLink: Boolean,
- pages: List[String]) extends LinkRenderer with WikiService {
+ /**
+ * Extends markedj Renderer for GitBucket
+ */
+ class GitBucketMarkedRenderer(options: Options,
+ repository: RepositoryService.RepositoryInfo,
+ enableWikiLink: Boolean,
+ enableRefsLink: Boolean,
+ enableAnchor: Boolean,
+ enableTaskList: Boolean,
+ hasWritePermission: Boolean,
+ pages: List[String])
+ (implicit val context: Context) extends Renderer(options) with LinkConverter with RequestCache {
- override def render(node: WikiLinkNode): Rendering = {
- if(enableWikiLink){
- try {
- val text = node.getText
- val (label, page) = if(text.contains('|')){
- val i = text.indexOf('|')
- (text.substring(0, i), text.substring(i + 1))
+ override def heading(text: String, level: Int, raw: String): String = {
+ val id = generateAnchorName(text)
+ val out = new StringBuilder()
+
+ out.append("" + + (if(escaped) code else escape(code, true)) + "" + } + + override def list(body: String, ordered: Boolean): String = { + var listType: String = null + if (ordered) { + listType = "ol" + } else { + listType = "ul" + } + if(body.contains("""class="task-list-item-checkbox"""")){ + "<" + listType + " class=\"task-list\">\n" + body + "\n" + } else { + "<" + listType + ">\n" + body + "\n" + } + } + + override def listitem(text: String): String = { + if(text.contains("""class="task-list-item-checkbox" """)){ + "
+ 