diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 4479460..49d184a 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -4,4 +4,3 @@ - If you can't find same question and report, send it to [gitter room](https://gitter.im/gitbucket/gitbucket) before raising an issue. - We can also support in Japaneses other than English at [gitter room for Japanese](https://gitter.im/gitbucket/gitbucket_ja). - Write an issue in English. At least, write subject in English. -- First priority of GitBucket is easy installation and reproduce GitHub behavior, so we might reject if your request is against it. diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index c59bec8..981e773 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -5,4 +5,4 @@ - [] verified that project is compiling - [] verified that tests are passing - [] squashed my commits as appropriate *(keep several commits if it is relevant to understand the PR)* -- [] [marked as closed](https://help.github.com/articles/closing-issues-via-commit-messages/) all issue ID that this PR should correct +- [] [marked as closed using commit message](https://help.github.com/articles/closing-issues-via-commit-messages/) all issue ID that this PR should correct diff --git a/.travis.yml b/.travis.yml index d4c6fff..889f1e2 100644 --- a/.travis.yml +++ b/.travis.yml @@ -8,4 +8,11 @@ - sudo apt-get install libaio1 - sudo /etc/init.d/mysql stop - sudo /etc/init.d/postgresql stop - +cache: + directories: + - $HOME/.ivy2/cache + - $HOME/.sbt/boot + - $HOME/.sbt/launchers + - $HOME/.coursier + - $HOME/.embedmysql + - $HOME/.embedpostgresql diff --git a/README.md b/README.md index b1ab25e..65e6bbe 100644 --- a/README.md +++ b/README.md @@ -2,30 +2,30 @@ ========= GitBucket is a Git platform powered by Scala offering: -- easy installation -- high extensibility by plugins -- API compatibility with Github +- Easy installation +- High extensibility by plugins +- API compatibility with GitHub Features -------- The current version of GitBucket provides a basic features below: - Public / Private Git repository (http and ssh access) -- Repository viewer and online file editing -- Wiki -- Issues / Pull request -- Email notification -- Simple user and group management with LDAP integration +- GitLFS support +- Repository viewer includes online file editor +- Issues, Pull request and Wiki for repositories +- Activity timeline and email notification +- Account and group management with LDAP integration - Plug-in system -If you want to try the development version of GitBucket, see [Developer's Guide](https://github.com/gitbucket/gitbucket/blob/master/doc/how_to_run.md). +If you want to try the development version of GitBucket, see the [Developer's Guide](https://github.com/gitbucket/gitbucket/blob/master/doc/how_to_run.md). Installation -------- -GitBucket requires **Java8**. You have to install beforehand when it's not installed. +GitBucket requires **Java8**. You have to install it if it is not already installed. -1. Download latest **gitbucket.war** from [the release page](https://github.com/gitbucket/gitbucket/releases) and run it by `java -jar gitbucket.war`. -2. Access `http://[hostname]:8080/` and logged in with **root** / **root**. +1. Download the latest **gitbucket.war** from [the releases page](https://github.com/gitbucket/gitbucket/releases) and run it by `java -jar gitbucket.war`. +2. Go to `http://[hostname]:8080/` and log in with **root** / **root**. You can specify following options: @@ -33,34 +33,47 @@ - `--prefix=[CONTEXTPATH]` - `--host=[HOSTNAME]` - `--gitbucket.home=[DATA_DIR]` +- `--temp_dir=[TEMP_DIR]` -Of course, you can also deploy gitbucket.war to a servlet container which supports Servlet 3.0 (like Jetty, Tomcat, JBoss, etc) +`TEMP_DIR` is used as the [temporary directory for the jetty application context](https://www.eclipse.org/jetty/documentation/9.3.x/ref-temporary-directories.html). This is the directory into which the `gitbucket.war` file is unpacked, the source files are compiled, etc. If given this parameter **must** match the path of an existing directory or the application will quit reporting an error; if not given the path used will be a `tmp` directory inside the gitbucket home. -About installation on Mac or Windows Server (with IIS), configuration of Apache or Nginx and also integration with other tools or services such as Jenkins or Slack, see [Wiki](https://github.com/gitbucket/gitbucket/wiki). +You can also deploy `gitbucket.war` to a servlet container which supports Servlet 3.0 (like Jetty, Tomcat, JBoss, etc) -To upgrade GitBucket, only replace gitbucket.war after stop GitBucket. All GitBucket data is stored in `HOME/.gitbucket` in default. So if you want to back up GitBucket data, copy this directory to the other disk. +For more information about installation on Mac or Windows Server (with IIS), or configuration of Apache or Nginx and also integration with other tools or services such as Jenkins or Slack, see [Wiki](https://github.com/gitbucket/gitbucket/wiki). -Plug-ins +To upgrade GitBucket, replace `gitbucket.war` with the new version, after stopping GitBucket. All GitBucket data is stored in `HOME/.gitbucket` by default. So if you want to back up GitBucket's data, copy this directory to the backup location. + +Plugins -------- -GitBucket has the plug-in system to extend GitBucket from outside of GitBucket. We are providing some official plug-ins: +GitBucket has a plug-in system to allow extensions to GitBucket. We provide some official plug-ins: - [gitbucket-gist-plugin](https://github.com/gitbucket/gitbucket-gist-plugin) - [gitbucket-emoji-plugin](https://github.com/gitbucket/gitbucket-emoji-plugin) -You can find more plugins made by community at [gitbucket community plugins](http://gitbucket-plugins.github.io/). +You can find more plugins made by the community at [GitBucket community plugins](http://gitbucket-plugins.github.io/). Support -------- -- If you have any question about GitBucket, send it to [gitter room](https://gitter.im/gitbucket/gitbucket) before raise an issue. -- Make sure check whether there is a same question or request in the past. -- When raise a new issue, write subject in **English** at least. -- We can also support in Japaneses other than English at [gitter room for Japanese](https://gitter.im/gitbucket/gitbucket_ja). -- First priority of GitBucket is easy installation and API compatibility with GitHub, so we might reject if your request is against it. +- If you have any questions about GitBucket, send it to the [gitter room](https://gitter.im/gitbucket/gitbucket) before opening an issue. +- Make sure check whether there is the same question or request in the past. +- When raise a new issue, write at least the subject in **English**. +- We can also provide support in Japanese at [gitter room for Japanese](https://gitter.im/gitbucket/gitbucket_ja). +- The first priority of GitBucket is easy installation and API compatibility with GitHub, so we might reject if your request is against it. Release Notes ------------- -## 4.8 - 23 Dec 2016 +### 4.9 - 29 Jan 2017 +- GitLFS support +- Template for issues and pull requests +- Manual label color editing +- Account description +- `--tmp-dir` option for standalone mode +- More APIs for issues + - [List issues for a repository](https://developer.github.com/v3/issues/#list-issues-for-a-repository) + - [Create an issue](https://developer.github.com/v3/issues/#create-an-issue) + +### 4.8 - 23 Dec 2016 - Search for repository names from the global header - Filter repositories on the sidebar of the dashboard - Search issues and wiki diff --git a/build.sbt b/build.sbt index 5ca0854..21be63d 100644 --- a/build.sbt +++ b/build.sbt @@ -1,6 +1,6 @@ val Organization = "io.github.gitbucket" val Name = "gitbucket" -val GitBucketVersion = "4.8" +val GitBucketVersion = "4.9.0" val ScalatraVersion = "2.5.0" val JettyVersion = "9.3.9.v20160517" @@ -22,8 +22,8 @@ ) libraryDependencies ++= Seq( "org.scala-lang.modules" %% "scala-java8-compat" % "0.8.0", - "org.eclipse.jgit" % "org.eclipse.jgit.http.server" % "4.1.2.201602141800-r", - "org.eclipse.jgit" % "org.eclipse.jgit.archive" % "4.1.2.201602141800-r", + "org.eclipse.jgit" % "org.eclipse.jgit.http.server" % "4.6.0.201612231935-r", + "org.eclipse.jgit" % "org.eclipse.jgit.archive" % "4.6.0.201612231935-r", "org.scalatra" %% "scalatra" % ScalatraVersion, "org.scalatra" %% "scalatra-json" % ScalatraVersion, "org.json4s" %% "json4s-jackson" % "3.5.0", @@ -37,7 +37,7 @@ "org.apache.sshd" % "apache-sshd" % "1.2.0", "org.apache.tika" % "tika-core" % "1.13", "com.github.takezoe" %% "blocking-slick" % "0.0.3", - "joda-time" % "joda-time" % "2.9.6", + "joda-time" % "joda-time" % "2.9.6", "com.novell.ldap" % "jldap" % "2009-10-07", "com.h2database" % "h2" % "1.4.192", "mysql" % "mysql-connector-java" % "5.1.39", @@ -47,14 +47,15 @@ "com.typesafe" % "config" % "1.3.0", "com.typesafe.akka" %% "akka-actor" % "2.4.12", "fr.brouillard.oss.security.xhub" % "xhub4j-core" % "1.0.0", - "com.github.bkromhout" % "java-diff-utils" % "2.1.1", - "org.cache2k" % "cache2k-all" % "1.0.0.CR1", + "com.github.bkromhout" % "java-diff-utils" % "2.1.1", + "org.cache2k" % "cache2k-all" % "1.0.0.CR1", "com.enragedginger" %% "akka-quartz-scheduler" % "1.6.0-akka-2.4.x" exclude("c3p0","c3p0"), + "net.coobird" % "thumbnailator" % "0.4.8", "org.eclipse.jetty" % "jetty-webapp" % JettyVersion % "provided", "javax.servlet" % "javax.servlet-api" % "3.1.0" % "provided", "junit" % "junit" % "4.12" % "test", "org.scalatra" %% "scalatra-scalatest" % ScalatraVersion % "test", - "com.wix" % "wix-embedded-mysql" % "1.0.3" % "test", + "com.wix" % "wix-embedded-mysql" % "2.1.4" % "test", "ru.yandex.qatools.embed" % "postgresql-embedded" % "1.14" % "test" ) @@ -166,9 +167,9 @@ log info s"built executable webapp ${outputFile}" outputFile } -publishTo <<= version { (v: String) => +publishTo := { val nexus = "https://oss.sonatype.org/" - if (v.trim.endsWith("SNAPSHOT")) Some("snapshots" at nexus + "content/repositories/snapshots") + if (version.value.trim.endsWith("SNAPSHOT")) Some("snapshots" at nexus + "content/repositories/snapshots") else Some("releases" at nexus + "service/local/staging/deploy/maven2") } publishMavenStyle := true diff --git a/contrib/linux/redhat/README.md b/contrib/linux/redhat/README.md index ddca44d..5c63178 100644 --- a/contrib/linux/redhat/README.md +++ b/contrib/linux/redhat/README.md @@ -3,6 +3,7 @@ RPM spec file and init script for Red Hat Enterprise Linux 6.x. To create RPM: + 1. Edit `../../gitbucket.conf` to suit. 2. Edit `gitbucket.init` to suit. 3. Edit `gitbucket.spec` to suit. diff --git a/project/plugins.sbt b/project/plugins.sbt index fb3bace..0ec8f00 100644 --- a/project/plugins.sbt +++ b/project/plugins.sbt @@ -5,3 +5,4 @@ addSbtPlugin("com.earldouglas" % "xsbt-web-plugin" % "2.1.1") addSbtPlugin("fi.gekkio.sbtplugins" % "sbt-jrebel-plugin" % "0.10.0") addSbtPlugin("com.typesafe.sbt" % "sbt-pgp" % "0.8.3") +addSbtPlugin("io.get-coursier" % "sbt-coursier" % "1.0.0-M15") diff --git a/project/project/plugins.sbt b/project/project/plugins.sbt new file mode 100644 index 0000000..5b48d85 --- /dev/null +++ b/project/project/plugins.sbt @@ -0,0 +1 @@ +addSbtPlugin("io.get-coursier" % "sbt-coursier" % "1.0.0-M15") diff --git a/src/main/java/JettyLauncher.java b/src/main/java/JettyLauncher.java index 1ac4187..d810941 100644 --- a/src/main/java/JettyLauncher.java +++ b/src/main/java/JettyLauncher.java @@ -12,6 +12,7 @@ int port = 8080; InetSocketAddress address = null; String contextPath = "/"; + String tmpDirPath=""; boolean forceHttps = false; for(String arg: args) { @@ -24,8 +25,13 @@ port = Integer.parseInt(dim[1]); } else if(dim[0].equals("--prefix")) { contextPath = dim[1]; + if(!contextPath.startsWith("/")){ + contextPath = "/" + contextPath; + } } else if(dim[0].equals("--gitbucket.home")){ System.setProperty("gitbucket.home", dim[1]); + } else if(dim[0].equals("--temp_dir")){ + tmpDirPath = dim[1]; } } } @@ -50,9 +56,21 @@ WebAppContext context = new WebAppContext(); - File tmpDir = new File(getGitBucketHome(), "tmp"); - if(!tmpDir.exists()){ - tmpDir.mkdirs(); + File tmpDir; + if(tmpDirPath.equals("")){ + tmpDir = new File(getGitBucketHome(), "tmp"); + if(!tmpDir.exists()){ + tmpDir.mkdirs(); + } + } else { + tmpDir = new File(tmpDirPath); + if(!tmpDir.exists()){ + throw new java.io.FileNotFoundException( + String.format("temp_dir \"%s\" not found", tmpDirPath)); + } else if(!tmpDir.isDirectory()) { + throw new IllegalArgumentException( + String.format("temp_dir \"%s\" is not a directory", tmpDirPath)); + } } context.setTempDirectory(tmpDir); diff --git a/src/main/resources/update/gitbucket-core_4.9.xml b/src/main/resources/update/gitbucket-core_4.9.xml new file mode 100644 index 0000000..afeffc2 --- /dev/null +++ b/src/main/resources/update/gitbucket-core_4.9.xml @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/src/main/scala/gitbucket/core/GitBucketCoreModule.scala b/src/main/scala/gitbucket/core/GitBucketCoreModule.scala index 3fadd2d..5bfe815 100644 --- a/src/main/scala/gitbucket/core/GitBucketCoreModule.scala +++ b/src/main/scala/gitbucket/core/GitBucketCoreModule.scala @@ -24,5 +24,8 @@ new SqlMigration("update/gitbucket-core_4.7.sql") ), new Version("4.7.1"), - new Version("4.8") + new Version("4.8"), + new Version("4.9.0", + new LiquibaseMigration("update/gitbucket-core_4.9.xml") + ) ) diff --git a/src/main/scala/gitbucket/core/api/CreateAnIssue.scala b/src/main/scala/gitbucket/core/api/CreateAnIssue.scala new file mode 100644 index 0000000..cb54652 --- /dev/null +++ b/src/main/scala/gitbucket/core/api/CreateAnIssue.scala @@ -0,0 +1,11 @@ +package gitbucket.core.api + +/** + * https://developer.github.com/v3/issues/#create-an-issue + */ +case class CreateAnIssue( + title: String, + body: Option[String], + assignees: List[String], + milestone: Option[Int], + labels: List[String]) diff --git a/src/main/scala/gitbucket/core/controller/AccountController.scala b/src/main/scala/gitbucket/core/controller/AccountController.scala index bfb9ccd..b16c747 100644 --- a/src/main/scala/gitbucket/core/controller/AccountController.scala +++ b/src/main/scala/gitbucket/core/controller/AccountController.scala @@ -29,10 +29,10 @@ with AccessTokenService with WebHookService with RepositoryCreationService => case class AccountNewForm(userName: String, password: String, fullName: String, mailAddress: String, - url: Option[String], fileId: Option[String]) + description: Option[String], url: Option[String], fileId: Option[String]) case class AccountEditForm(password: Option[String], fullName: String, mailAddress: String, - url: Option[String], fileId: Option[String], clearImage: Boolean) + description: Option[String], url: Option[String], fileId: Option[String], clearImage: Boolean) case class SshKeyForm(title: String, publicKey: String) @@ -43,6 +43,7 @@ "password" -> trim(label("Password" , text(required, maxlength(20)))), "fullName" -> trim(label("Full Name" , text(required, maxlength(100)))), "mailAddress" -> trim(label("Mail Address" , text(required, maxlength(100), uniqueMailAddress()))), + "description" -> trim(label("bio" , optional(text()))), "url" -> trim(label("URL" , optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" , optional(text()))) )(AccountNewForm.apply) @@ -51,6 +52,7 @@ "password" -> trim(label("Password" , optional(text(maxlength(20))))), "fullName" -> trim(label("Full Name" , text(required, maxlength(100)))), "mailAddress" -> trim(label("Mail Address" , text(required, maxlength(100), uniqueMailAddress("userName")))), + "description" -> trim(label("bio" , optional(text()))), "url" -> trim(label("URL" , optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" , optional(text()))), "clearImage" -> trim(label("Clear image" , boolean())) @@ -65,11 +67,12 @@ "note" -> trim(label("Token", text(required, maxlength(100)))) )(PersonalTokenForm.apply) - case class NewGroupForm(groupName: String, url: Option[String], fileId: Option[String], members: String) - case class EditGroupForm(groupName: String, url: Option[String], fileId: Option[String], members: String, clearImage: Boolean) + case class NewGroupForm(groupName: String, description: Option[String], url: Option[String], fileId: Option[String], members: String) + case class EditGroupForm(groupName: String, description: Option[String], url: Option[String], fileId: Option[String], members: String, clearImage: Boolean) val newGroupForm = mapping( "groupName" -> trim(label("Group name" ,text(required, maxlength(100), identifier, uniqueUserName, reservedNames))), + "description" -> trim(label("Group description", optional(text()))), "url" -> trim(label("URL" ,optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" ,optional(text()))), "members" -> trim(label("Members" ,text(required, members))) @@ -77,6 +80,7 @@ val editGroupForm = mapping( "groupName" -> trim(label("Group name" ,text(required, maxlength(100), identifier))), + "description" -> trim(label("Group description", optional(text()))), "url" -> trim(label("URL" ,optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" ,optional(text()))), "members" -> trim(label("Members" ,text(required, members))), @@ -167,6 +171,7 @@ password = form.password.map(sha1).getOrElse(account.password), fullName = form.fullName, mailAddress = form.mailAddress, + description = form.description, url = form.url)) updateImage(userName, form.fileId, form.clearImage) @@ -266,7 +271,7 @@ post("/register", newForm){ form => if(context.settings.allowAccountRegistration){ - createAccount(form.userName, sha1(form.password), form.fullName, form.mailAddress, false, form.url) + createAccount(form.userName, sha1(form.password), form.fullName, form.mailAddress, false, form.description, form.url) updateImage(form.userName, form.fileId, false) redirect("/signin") } else NotFound() @@ -277,7 +282,7 @@ }) post("/groups/new", newGroupForm)(usersOnly { form => - createGroup(form.groupName, form.url) + createGroup(form.groupName, form.description, form.url) updateGroupMembers(form.groupName, form.members.split(",").map { _.split(":") match { case Array(userName, isManager) => (userName, isManager.toBoolean) @@ -315,7 +320,7 @@ } }.toList){ case (groupName, members) => getAccountByUserName(groupName, true).map { account => - updateGroup(groupName, form.url, false) + updateGroup(groupName, form.description, form.url, false) // Update GROUP_MEMBER updateGroupMembers(form.groupName, members) diff --git a/src/main/scala/gitbucket/core/controller/ApiController.scala b/src/main/scala/gitbucket/core/controller/ApiController.scala index d118fa4..794b451 100644 --- a/src/main/scala/gitbucket/core/controller/ApiController.scala +++ b/src/main/scala/gitbucket/core/controller/ApiController.scala @@ -21,10 +21,12 @@ with ProtectedBranchService with IssuesService with LabelsService + with MilestonesService with PullRequestService with CommitsService with CommitStatusService with RepositoryCreationService + with IssueCreationService with HandleCommentService with WebHookService with WebHookPullRequestService @@ -44,9 +46,11 @@ with ProtectedBranchService with IssuesService with LabelsService + with MilestonesService with PullRequestService with CommitStatusService with RepositoryCreationService + with IssueCreationService with HandleCommentService with OwnerAuthenticator with UsersAuthenticator @@ -56,6 +60,13 @@ with WritableUsersAuthenticator => /** + * 404 for non-implemented api + */ + get("/api/v3/*") { + NotFound() + } + + /** * https://developer.github.com/v3/#root-endpoint */ get("/api/v3/") { @@ -175,7 +186,7 @@ using(Git.open(getRepositoryDir(params("owner"), params("repo")))) { git => //JsonFormat( (revstr, git.getRepository().resolve(revstr)) ) // getRef is deprecated by jgit-4.2. use exactRef() or findRef() - val sha = git.getRepository().getRef(revstr).getObjectId().name() + val sha = git.getRepository().exactRef(revstr).getObjectId().name() JsonFormat(ApiRef(revstr, ApiObject(sha))) } }) @@ -284,6 +295,32 @@ } /** + * https://developer.github.com/v3/issues/#list-issues-for-a-repository + */ + get("/api/v3/repos/:owner/:repository/issues")(referrersOnly { repository => + val page = IssueSearchCondition.page(request) + // TODO: more api spec condition + val condition = IssueSearchCondition(request) + val baseOwner = getAccountByUserName(repository.owner).get + + val issues: List[(Issue, Account)] = + searchIssueByApi( + condition = condition, + offset = (page - 1) * PullRequestLimit, + limit = PullRequestLimit, + repos = repository.owner -> repository.name + ) + + JsonFormat(issues.map { case (issue, issueUser) => + ApiIssue( + issue = issue, + repositoryName = RepositoryName(repository), + user = ApiUser(issueUser) + ) + }) + }) + + /** * https://developer.github.com/v3/issues/#get-a-single-issue */ get("/api/v3/repos/:owner/:repository/issues/:id")(referrersOnly { repository => @@ -297,6 +334,29 @@ }) /** + * https://developer.github.com/v3/issues/#create-an-issue + */ + post("/api/v3/repos/:owner/:repository/issues")(readableUsersOnly { repository => + if(isIssueEditable(repository)){ // TODO Should this check is provided by authenticator? + (for{ + data <- extractFromJsonBody[CreateAnIssue] + loginAccount <- context.loginAccount + } yield { + val milestone = data.milestone.flatMap(getMilestone(repository.owner, repository.name, _)) + val issue = createIssue( + repository, + data.title, + data.body, + data.assignees.headOption, + milestone.map(_.milestoneId), + data.labels, + loginAccount) + JsonFormat(ApiIssue(issue, RepositoryName(repository), ApiUser(loginAccount))) + }) getOrElse NotFound() + } else Unauthorized() + }) + + /** * https://developer.github.com/v3/issues/comments/#list-comments-on-an-issue */ get("/api/v3/repos/:owner/:repository/issues/:id/comments")(referrersOnly { repository => diff --git a/src/main/scala/gitbucket/core/controller/ControllerBase.scala b/src/main/scala/gitbucket/core/controller/ControllerBase.scala index eeedd1c..8e42e37 100644 --- a/src/main/scala/gitbucket/core/controller/ControllerBase.scala +++ b/src/main/scala/gitbucket/core/controller/ControllerBase.scala @@ -9,7 +9,6 @@ import gitbucket.core.util._ import io.github.gitbucket.scalatra.forms._ -import org.apache.commons.io.FileUtils import org.json4s._ import org.scalatra._ import org.scalatra.i18n._ @@ -20,6 +19,8 @@ import scala.util.Try +import net.coobird.thumbnailator.Thumbnails + /** * Provides generic features for controller implementations. @@ -57,7 +58,7 @@ // Redirect to dashboard httpResponse.sendRedirect(baseUrl + "/") } - } else if(path.startsWith("/git/")){ + } else if(path.startsWith("/git/") || path.startsWith("/git-lfs/")){ // Git repository chain.doFilter(request, response) } else { @@ -225,10 +226,13 @@ } else { fileId.map { fileId => val filename = "avatar." + FileUtil.getExtension(session.getAndRemove(Keys.Session.Upload(fileId)).get) - FileUtils.moveFile( - new java.io.File(getTemporaryDir(session.getId), fileId), - new java.io.File(getUserUploadDir(userName), filename) - ) + val uploadDir = getUserUploadDir(userName) + if(!uploadDir.exists){ + uploadDir.mkdirs() + } + Thumbnails.of(new java.io.File(getTemporaryDir(session.getId), fileId)) + .size(324, 324) + .toFile(new java.io.File(uploadDir, filename)) updateAvatarImage(userName, Some(filename)) } } diff --git a/src/main/scala/gitbucket/core/controller/FileUploadController.scala b/src/main/scala/gitbucket/core/controller/FileUploadController.scala index d264341..3296903 100644 --- a/src/main/scala/gitbucket/core/controller/FileUploadController.scala +++ b/src/main/scala/gitbucket/core/controller/FileUploadController.scala @@ -10,7 +10,6 @@ import org.eclipse.jgit.api.Git import org.eclipse.jgit.dircache.DirCache import org.eclipse.jgit.lib.{FileMode, Constants} -import org.scalatra import org.scalatra._ import org.scalatra.servlet.{MultipartConfig, FileUploadSupport, FileItem} import org.apache.commons.io.{IOUtils, FileUtils} diff --git a/src/main/scala/gitbucket/core/controller/IndexController.scala b/src/main/scala/gitbucket/core/controller/IndexController.scala index 6b4db67..1cd4282 100644 --- a/src/main/scala/gitbucket/core/controller/IndexController.scala +++ b/src/main/scala/gitbucket/core/controller/IndexController.scala @@ -5,7 +5,7 @@ import gitbucket.core.service._ import gitbucket.core.util.Implicits._ import gitbucket.core.util.ControlUtil._ -import gitbucket.core.util.{Keys, LDAPUtil, ReferrerAuthenticator, StringUtil, UsersAuthenticator} +import gitbucket.core.util.{Keys, LDAPUtil, ReferrerAuthenticator, UsersAuthenticator} import io.github.gitbucket.scalatra.forms._ import org.scalatra.Ok @@ -49,13 +49,18 @@ if(redirect.isDefined && redirect.get.startsWith("/")){ flash += Keys.Flash.Redirect -> redirect.get } - gitbucket.core.html.signin() + gitbucket.core.html.signin(flash.get("userName"), flash.get("password"), flash.get("error")) } post("/signin", signinForm){ form => authenticate(context.settings, form.userName, form.password) match { case Some(account) => signin(account) - case None => redirect("/signin") + case None => { + flash += "userName" -> form.userName + flash += "password" -> form.password + flash += "error" -> "Sorry, your Username and/or Password is incorrect. Please try again." + redirect("/signin") + } } } diff --git a/src/main/scala/gitbucket/core/controller/IssuesController.scala b/src/main/scala/gitbucket/core/controller/IssuesController.scala index 4222dba..f55db68 100644 --- a/src/main/scala/gitbucket/core/controller/IssuesController.scala +++ b/src/main/scala/gitbucket/core/controller/IssuesController.scala @@ -2,7 +2,6 @@ import gitbucket.core.issues.html import gitbucket.core.service.IssuesService._ -import gitbucket.core.service.RepositoryService.RepositoryInfo import gitbucket.core.service._ import gitbucket.core.util.ControlUtil._ import gitbucket.core.util.Implicits._ @@ -10,7 +9,7 @@ import gitbucket.core.view import gitbucket.core.view.Markdown import io.github.gitbucket.scalatra.forms._ -import org.scalatra.Ok +import org.scalatra.{BadRequest, Ok} class IssuesController extends IssuesControllerBase @@ -21,6 +20,7 @@ with MilestonesService with ActivityService with HandleCommentService + with IssueCreationService with ReadableUsersAuthenticator with ReferrerAuthenticator with WritableUsersAuthenticator @@ -36,6 +36,7 @@ with MilestonesService with ActivityService with HandleCommentService + with IssueCreationService with ReadableUsersAuthenticator with ReferrerAuthenticator with WritableUsersAuthenticator @@ -91,67 +92,39 @@ getAssignableUserNames(owner, name), getMilestonesWithIssueCount(owner, name), getLabels(owner, name), - isEditable(repository), - isManageable(repository), + isIssueEditable(repository), + isIssueManageable(repository), repository) } getOrElse NotFound() } }) get("/:owner/:repository/issues/new")(readableUsersOnly { repository => - if(isEditable(repository)){ // TODO Should this check is provided by authenticator? + if(isIssueEditable(repository)){ // TODO Should this check is provided by authenticator? defining(repository.owner, repository.name){ case (owner, name) => html.create( getAssignableUserNames(owner, name), getMilestones(owner, name), getLabels(owner, name), - isManageable(repository), + isIssueManageable(repository), + getContentTemplate(repository, "ISSUE_TEMPLATE"), repository) } } else Unauthorized() }) post("/:owner/:repository/issues/new", issueCreateForm)(readableUsersOnly { (form, repository) => - if(isEditable(repository)){ // TODO Should this check is provided by authenticator? - defining(repository.owner, repository.name){ case (owner, name) => - val manageable = isManageable(repository) - val userName = context.loginAccount.get.userName + if(isIssueEditable(repository)){ // TODO Should this check is provided by authenticator? + val issue = createIssue( + repository, + form.title, + form.content, + form.assignedUserName, + form.milestoneId, + form.labelNames.toArray.flatMap(_.split(",")), + context.loginAccount.get) - // insert issue - val issueId = createIssue(owner, name, userName, form.title, form.content, - if (manageable) form.assignedUserName else None, - if (manageable) form.milestoneId else None) - - // insert labels - if (manageable) { - form.labelNames.map { value => - val labels = getLabels(owner, name) - value.split(",").foreach { labelName => - labels.find(_.labelName == labelName).map { label => - registerIssueLabel(owner, name, issueId, label.labelId) - } - } - } - } - - // record activity - recordCreateIssueActivity(owner, name, userName, issueId, form.title) - - getIssue(owner, name, issueId.toString).foreach { issue => - // extract references and create refer comment - createReferComment(owner, name, issue, form.title + " " + form.content.getOrElse(""), context.loginAccount.get) - - // call web hooks - callIssuesWebHook("opened", repository, issue, context.baseUrl, context.loginAccount.get) - - // notifications - Notifier().toNotify(repository, issue, form.content.getOrElse("")) { - Notifier.msgIssue(s"${context.baseUrl}/${owner}/${name}/issues/${issueId}") - } - } - - redirect(s"/${owner}/${name}/issues/${issueId}") - } + redirect(s"/${issue.userName}/${issue.repositoryName}/issues/${issue.issueId}") } else Unauthorized() }) @@ -327,7 +300,7 @@ handleComment(issue, None, repository, Some("close")) } } - case _ => // TODO BadRequest + case _ => BadRequest() } } }) @@ -398,27 +371,8 @@ countIssue(condition.copy(state = "closed"), false, owner -> repoName), condition, repository, - isEditable(repository), - isManageable(repository)) - } - } - - /** - * Tests whether an logged-in user can manage issues. - */ - private def isManageable(repository: RepositoryInfo)(implicit context: Context): Boolean = { - hasDeveloperRole(repository.owner, repository.name, context.loginAccount) - } - - /** - * Tests whether an logged-in user can post issues. - */ - private def isEditable(repository: RepositoryInfo)(implicit context: Context): Boolean = { - repository.repository.options.issuesOption match { - case "ALL" => !repository.repository.isPrivate && context.loginAccount.isDefined - case "PUBLIC" => hasGuestRole(repository.owner, repository.name, context.loginAccount) - case "PRIVATE" => hasDeveloperRole(repository.owner, repository.name, context.loginAccount) - case "DISABLE" => false + isIssueEditable(repository), + isIssueManageable(repository)) } } @@ -428,5 +382,4 @@ private def isEditableContent(owner: String, repository: String, author: String)(implicit context: Context): Boolean = { hasDeveloperRole(owner, repository, context.loginAccount) || author == context.loginAccount.get.userName } - } diff --git a/src/main/scala/gitbucket/core/controller/PullRequestsController.scala b/src/main/scala/gitbucket/core/controller/PullRequestsController.scala index 671d9f1..8609421 100644 --- a/src/main/scala/gitbucket/core/controller/PullRequestsController.scala +++ b/src/main/scala/gitbucket/core/controller/PullRequestsController.scala @@ -368,6 +368,7 @@ forkedId, oldId.getName, newId.getName, + getContentTemplate(originRepository, "PULL_REQUEST_TEMPLATE"), forkedRepository, originRepository, forkedRepository, @@ -424,7 +425,7 @@ if(editable) { val loginUserName = context.loginAccount.get.userName - val issueId = createIssue( + val issueId = insertIssue( owner = repository.owner, repository = repository.name, loginUser = loginUserName, diff --git a/src/main/scala/gitbucket/core/controller/RepositorySettingsController.scala b/src/main/scala/gitbucket/core/controller/RepositorySettingsController.scala index b0b30f5..6a349d9 100644 --- a/src/main/scala/gitbucket/core/controller/RepositorySettingsController.scala +++ b/src/main/scala/gitbucket/core/controller/RepositorySettingsController.scala @@ -338,6 +338,7 @@ FileUtils.deleteDirectory(getRepositoryDir(repository.owner, repository.name)) FileUtils.deleteDirectory(getWikiRepositoryDir(repository.owner, repository.name)) FileUtils.deleteDirectory(getTemporaryDir(repository.owner, repository.name)) + FileUtils.deleteDirectory(getLfsDir(repository.owner, repository.name)) } redirect(s"/${repository.owner}") }) diff --git a/src/main/scala/gitbucket/core/controller/RepositoryViewerController.scala b/src/main/scala/gitbucket/core/controller/RepositoryViewerController.scala index d08c0fe..b21b03f 100644 --- a/src/main/scala/gitbucket/core/controller/RepositoryViewerController.scala +++ b/src/main/scala/gitbucket/core/controller/RepositoryViewerController.scala @@ -1,6 +1,7 @@ package gitbucket.core.controller -import javax.servlet.http.{HttpServletResponse, HttpServletRequest} +import java.io.FileInputStream +import javax.servlet.http.{HttpServletRequest, HttpServletResponse} import gitbucket.core.plugin.PluginRegistry import gitbucket.core.repo.html @@ -16,9 +17,8 @@ import gitbucket.core.service.WebHookService._ import gitbucket.core.view import gitbucket.core.view.helpers - import io.github.gitbucket.scalatra.forms._ -import org.apache.commons.io.FileUtils +import org.apache.commons.io.IOUtils import org.eclipse.jgit.api.{ArchiveCommand, Git} import org.eclipse.jgit.archive.{TgzFormat, ZipFormat} import org.eclipse.jgit.dircache.DirCache @@ -255,13 +255,9 @@ val (id, path) = repository.splitPath(multiParams("splat").head) using(Git.open(getRepositoryDir(repository.owner, repository.name))){ git => val revCommit = JGitUtil.getRevCommitFromId(git, git.getRepository.resolve(id)) - getPathObjectId(git, path, revCommit).flatMap { objectId => - JGitUtil.getObjectLoaderFromId(git, objectId){ loader => - contentType = FileUtil.getMimeType(path) - response.setContentLength(loader.getSize.toInt) - loader.copyTo(response.outputStream) - () - } + + getPathObjectId(git, path, revCommit).map { objectId => + responseRawFile(git, objectId, path, repository) } getOrElse NotFound() } }) @@ -277,23 +273,62 @@ getPathObjectId(git, path, revCommit).map { objectId => if(raw){ // Download (This route is left for backword compatibility) - JGitUtil.getObjectLoaderFromId(git, objectId){ loader => - contentType = FileUtil.getMimeType(path) - response.setContentLength(loader.getSize.toInt) - loader.copyTo(response.outputStream) - () - } getOrElse NotFound() + responseRawFile(git, objectId, path, repository) } else { html.blob(id, repository, path.split("/").toList, JGitUtil.getContentInfo(git, path, objectId), new JGitUtil.CommitInfo(JGitUtil.getLastModifiedCommit(git, revCommit, path)), hasDeveloperRole(repository.owner, repository.name, context.loginAccount), - request.paths(2) == "blame") + request.paths(2) == "blame", + isLfsFile(git, objectId)) } } getOrElse NotFound() } }) + private def isLfsFile(git: Git, objectId: ObjectId): Boolean = { + JGitUtil.getObjectLoaderFromId(git, objectId){ loader => + if(loader.isLarge){ + false + } else { + new String(loader.getCachedBytes, "UTF-8").startsWith("version https://git-lfs.github.com/spec/v1") + } + }.getOrElse(false) + } + + private def responseRawFile(git: Git, objectId: ObjectId, path: String, + repository: RepositoryService.RepositoryInfo): Unit = { + JGitUtil.getObjectLoaderFromId(git, objectId){ loader => + contentType = FileUtil.getMimeType(path) + + if(loader.isLarge){ + response.setContentLength(loader.getSize.toInt) + loader.copyTo(response.outputStream) + } else { + val bytes = loader.getCachedBytes + val text = new String(bytes, "UTF-8") + + if(text.startsWith("version https://git-lfs.github.com/spec/v1")){ + // LFS objects + val attrs = text.split("\n").map { line => + val dim = line.split(" ") + dim(0) -> dim(1) + }.toMap + + response.setContentLength(attrs("size").toInt) + val oid = attrs("oid").split(":")(1) + + using(new FileInputStream(FileUtil.getLfsFilePath(repository.owner, repository.name, oid))){ in => + IOUtils.copy(in, response.getOutputStream) + } + } else { + response.setContentLength(loader.getSize.toInt) + response.getOutputStream.write(bytes) + } + } + } + } + get("/:owner/:repository/blame/*"){ blobRoute.action() } @@ -683,11 +718,6 @@ private def archiveRepository(name: String, suffix: String, repository: RepositoryService.RepositoryInfo): Unit = { val revision = name.stripSuffix(suffix) - val workDir = getDownloadWorkDir(repository.owner, repository.name, session.getId) - if(workDir.exists) { - FileUtils.deleteDirectory(workDir) - } - workDir.mkdirs val filename = repository.name + "-" + (if(revision.length == 40) revision.substring(0, 10) else revision).replace('/', '_') + suffix @@ -701,7 +731,7 @@ git.archive .setFormat(suffix.tail) - .setTree(revCommit.getTree) + .setTree(revCommit) .setOutputStream(response.getOutputStream) .call() } diff --git a/src/main/scala/gitbucket/core/controller/SystemSettingsController.scala b/src/main/scala/gitbucket/core/controller/SystemSettingsController.scala index bfca858..801b873 100644 --- a/src/main/scala/gitbucket/core/controller/SystemSettingsController.scala +++ b/src/main/scala/gitbucket/core/controller/SystemSettingsController.scala @@ -41,6 +41,7 @@ "user" -> trim(label("SMTP User", optional(text()))), "password" -> trim(label("SMTP Password", optional(text()))), "ssl" -> trim(label("Enable SSL", optional(boolean()))), + "starttls" -> trim(label("Enable STARTTLS", optional(boolean()))), "fromAddress" -> trim(label("FROM Address", optional(text()))), "fromName" -> trim(label("FROM Name", optional(text()))) )(Smtp.apply)), @@ -77,6 +78,7 @@ "user" -> trim(label("SMTP User", optional(text()))), "password" -> trim(label("SMTP Password", optional(text()))), "ssl" -> trim(label("Enable SSL", optional(boolean()))), + "starttls" -> trim(label("Enable STARTTLS", optional(boolean()))), "fromAddress" -> trim(label("FROM Address", optional(text()))), "fromName" -> trim(label("FROM Name", optional(text()))) )(Smtp.apply), @@ -89,16 +91,16 @@ case class NewUserForm(userName: String, password: String, fullName: String, mailAddress: String, isAdmin: Boolean, - url: Option[String], fileId: Option[String]) + description: Option[String], url: Option[String], fileId: Option[String]) case class EditUserForm(userName: String, password: Option[String], fullName: String, - mailAddress: String, isAdmin: Boolean, url: Option[String], + mailAddress: String, isAdmin: Boolean, description: Option[String], url: Option[String], fileId: Option[String], clearImage: Boolean, isRemoved: Boolean) - case class NewGroupForm(groupName: String, url: Option[String], fileId: Option[String], + case class NewGroupForm(groupName: String, description: Option[String], url: Option[String], fileId: Option[String], members: String) - case class EditGroupForm(groupName: String, url: Option[String], fileId: Option[String], + case class EditGroupForm(groupName: String, description: Option[String], url: Option[String], fileId: Option[String], members: String, clearImage: Boolean, isRemoved: Boolean) @@ -108,6 +110,7 @@ "fullName" -> trim(label("Full Name" ,text(required, maxlength(100)))), "mailAddress" -> trim(label("Mail Address" ,text(required, maxlength(100), uniqueMailAddress()))), "isAdmin" -> trim(label("User Type" ,boolean())), + "description" -> trim(label("bio" ,optional(text()))), "url" -> trim(label("URL" ,optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" ,optional(text()))) )(NewUserForm.apply) @@ -118,6 +121,7 @@ "fullName" -> trim(label("Full Name" ,text(required, maxlength(100)))), "mailAddress" -> trim(label("Mail Address" ,text(required, maxlength(100), uniqueMailAddress("userName")))), "isAdmin" -> trim(label("User Type" ,boolean())), + "description" -> trim(label("bio" ,optional(text()))), "url" -> trim(label("URL" ,optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" ,optional(text()))), "clearImage" -> trim(label("Clear image" ,boolean())), @@ -126,6 +130,7 @@ val newGroupForm = mapping( "groupName" -> trim(label("Group name" ,text(required, maxlength(100), identifier, uniqueUserName, reservedNames))), + "description" -> trim(label("Group description", optional(text()))), "url" -> trim(label("URL" ,optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" ,optional(text()))), "members" -> trim(label("Members" ,text(required, members))) @@ -133,6 +138,7 @@ val editGroupForm = mapping( "groupName" -> trim(label("Group name" ,text(required, maxlength(100), identifier))), + "description" -> trim(label("Group description", optional(text()))), "url" -> trim(label("URL" ,optional(text(maxlength(200))))), "fileId" -> trim(label("File ID" ,optional(text()))), "members" -> trim(label("Members" ,text(required, members))), @@ -164,7 +170,8 @@ post("/admin/system/sendmail", sendMailForm)(adminOnly { form => try { new Mailer(form.smtp).send(form.testAddress, - "Test message from GitBucket", "This is a test message from GitBucket.") + "Test message from GitBucket", "This is a test message from GitBucket.", + context.loginAccount.get) "Test mail has been sent to: " + form.testAddress @@ -193,7 +200,7 @@ }) post("/admin/users/_newuser", newUserForm)(adminOnly { form => - createAccount(form.userName, sha1(form.password), form.fullName, form.mailAddress, form.isAdmin, form.url) + createAccount(form.userName, sha1(form.password), form.fullName, form.mailAddress, form.isAdmin, form.description, form.url) updateImage(form.userName, form.fileId, false) redirect("/admin/users") }) @@ -227,6 +234,7 @@ fullName = form.fullName, mailAddress = form.mailAddress, isAdmin = form.isAdmin, + description = form.description, url = form.url, isRemoved = form.isRemoved)) @@ -241,7 +249,7 @@ }) post("/admin/users/_newgroup", newGroupForm)(adminOnly { form => - createGroup(form.groupName, form.url) + createGroup(form.groupName, form.description, form.url) updateGroupMembers(form.groupName, form.members.split(",").map { _.split(":") match { case Array(userName, isManager) => (userName, isManager.toBoolean) @@ -264,7 +272,7 @@ } }.toList){ case (groupName, members) => getAccountByUserName(groupName, true).map { account => - updateGroup(groupName, form.url, form.isRemoved) + updateGroup(groupName, form.url, form.description, form.isRemoved) if(form.isRemoved){ // Remove from GROUP_MEMBER diff --git a/src/main/scala/gitbucket/core/model/Account.scala b/src/main/scala/gitbucket/core/model/Account.scala index 57673c0..1f4931e 100644 --- a/src/main/scala/gitbucket/core/model/Account.scala +++ b/src/main/scala/gitbucket/core/model/Account.scala @@ -19,7 +19,8 @@ val image = column[String]("IMAGE") val groupAccount = column[Boolean]("GROUP_ACCOUNT") val removed = column[Boolean]("REMOVED") - def * = (userName, fullName, mailAddress, password, isAdmin, url.?, registeredDate, updatedDate, lastLoginDate.?, image.?, groupAccount, removed) <> (Account.tupled, Account.unapply) + val description = column[String]("DESCRIPTION") + def * = (userName, fullName, mailAddress, password, isAdmin, url.?, registeredDate, updatedDate, lastLoginDate.?, image.?, groupAccount, removed, description.?) <> (Account.tupled, Account.unapply) } } @@ -35,5 +36,6 @@ lastLoginDate: Option[java.util.Date], image: Option[String], isGroupAccount: Boolean, - isRemoved: Boolean + isRemoved: Boolean, + description: Option[String] ) diff --git a/src/main/scala/gitbucket/core/model/CommitStatus.scala b/src/main/scala/gitbucket/core/model/CommitStatus.scala index f6f773b..4bb01e0 100644 --- a/src/main/scala/gitbucket/core/model/CommitStatus.scala +++ b/src/main/scala/gitbucket/core/model/CommitStatus.scala @@ -1,8 +1,5 @@ package gitbucket.core.model -//import scala.slick.lifted.MappedTo -//import scala.slick.jdbc._ - trait CommitStatusComponent extends TemplateComponent { self: Profile => import profile.api._ import self._ diff --git a/src/main/scala/gitbucket/core/service/AccountService.scala b/src/main/scala/gitbucket/core/service/AccountService.scala index 941a587..eb13645 100644 --- a/src/main/scala/gitbucket/core/service/AccountService.scala +++ b/src/main/scala/gitbucket/core/service/AccountService.scala @@ -14,13 +14,20 @@ private val logger = LoggerFactory.getLogger(classOf[AccountService]) - def authenticate(settings: SystemSettings, userName: String, password: String)(implicit s: Session): Option[Account] = - if(settings.ldapAuthentication){ + def authenticate(settings: SystemSettings, userName: String, password: String)(implicit s: Session): Option[Account] = { + val account = if (settings.ldapAuthentication) { ldapAuthentication(settings, userName, password) } else { defaultAuthentication(userName, password) } + if(account.isEmpty){ + logger.info(s"Failed to authenticate: $userName") + } + + account + } + /** * Authenticate by internal database. */ @@ -61,14 +68,14 @@ defaultAuthentication(userName, password) } case None => { - createAccount(ldapUserInfo.userName, "", ldapUserInfo.fullName, ldapUserInfo.mailAddress, false, None) + createAccount(ldapUserInfo.userName, "", ldapUserInfo.fullName, ldapUserInfo.mailAddress, false, None, None) getAccountByUserName(ldapUserInfo.userName) } } } } case Left(errorMessage) => { - logger.info(s"LDAP Authentication Failed: ${errorMessage}") + logger.info(s"LDAP error: ${errorMessage}") defaultAuthentication(userName, password) } } @@ -103,7 +110,7 @@ } else false } - def createAccount(userName: String, password: String, fullName: String, mailAddress: String, isAdmin: Boolean, url: Option[String]) + def createAccount(userName: String, password: String, fullName: String, mailAddress: String, isAdmin: Boolean, description: Option[String], url: Option[String]) (implicit s: Session): Unit = Accounts insert Account( userName = userName, @@ -117,12 +124,13 @@ lastLoginDate = None, image = None, isGroupAccount = false, - isRemoved = false) + isRemoved = false, + description = description) def updateAccount(account: Account)(implicit s: Session): Unit = Accounts .filter { a => a.userName === account.userName.bind } - .map { a => (a.password, a.fullName, a.mailAddress, a.isAdmin, a.url.?, a.registeredDate, a.updatedDate, a.lastLoginDate.?, a.removed) } + .map { a => (a.password, a.fullName, a.mailAddress, a.isAdmin, a.url.?, a.registeredDate, a.updatedDate, a.lastLoginDate.?, a.removed, a.description.?) } .update ( account.password, account.fullName, @@ -132,7 +140,8 @@ account.registeredDate, currentDate, account.lastLoginDate, - account.isRemoved) + account.isRemoved, + account.description) def updateAvatarImage(userName: String, image: Option[String])(implicit s: Session): Unit = Accounts.filter(_.userName === userName.bind).map(_.image.?).update(image) @@ -140,7 +149,7 @@ def updateLastLoginDate(userName: String)(implicit s: Session): Unit = Accounts.filter(_.userName === userName.bind).map(_.lastLoginDate).update(currentDate) - def createGroup(groupName: String, url: Option[String])(implicit s: Session): Unit = + def createGroup(groupName: String, description: Option[String], url: Option[String])(implicit s: Session): Unit = Accounts insert Account( userName = groupName, password = "", @@ -153,10 +162,13 @@ lastLoginDate = None, image = None, isGroupAccount = true, - isRemoved = false) + isRemoved = false, + description = description) - def updateGroup(groupName: String, url: Option[String], removed: Boolean)(implicit s: Session): Unit = - Accounts.filter(_.userName === groupName.bind).map(t => t.url.? -> t.removed).update(url, removed) + def updateGroup(groupName: String, description: Option[String], url: Option[String], removed: Boolean)(implicit s: Session): Unit = + Accounts.filter(_.userName === groupName.bind) + .map(t => (t.url.?, t.description.?, t.removed)) + .update(url, description, removed) def updateGroupMembers(groupName: String, members: List[(String, Boolean)])(implicit s: Session): Unit = { GroupMembers.filter(_.groupName === groupName.bind).delete diff --git a/src/main/scala/gitbucket/core/service/HandleCommentService.scala b/src/main/scala/gitbucket/core/service/HandleCommentService.scala index 225766b..8c96042 100644 --- a/src/main/scala/gitbucket/core/service/HandleCommentService.scala +++ b/src/main/scala/gitbucket/core/service/HandleCommentService.scala @@ -17,76 +17,77 @@ */ def handleComment(issue: Issue, content: Option[String], repository: RepositoryService.RepositoryInfo, actionOpt: Option[String]) (implicit context: Context, s: Session) = { + context.loginAccount.flatMap { loginAccount => + defining(repository.owner, repository.name){ case (owner, name) => + val userName = loginAccount.userName - defining(repository.owner, repository.name){ case (owner, name) => - val userName = context.loginAccount.get.userName - - val (action, recordActivity) = actionOpt - .collect { - case "close" if(!issue.closed) => true -> - (Some("close") -> Some(if(issue.isPullRequest) recordClosePullRequestActivity _ else recordCloseIssueActivity _)) - case "reopen" if(issue.closed) => false -> - (Some("reopen") -> Some(recordReopenIssueActivity _)) - } - .map { case (closed, t) => - updateClosed(owner, name, issue.issueId, closed) - t - } - .getOrElse(None -> None) - - val commentId = (content, action) match { - case (None, None) => None - case (None, Some(action)) => Some(createComment(owner, name, userName, issue.issueId, action.capitalize, action)) - case (Some(content), _) => Some(createComment(owner, name, userName, issue.issueId, content, action.map(_+ "_comment").getOrElse("comment"))) - } - - // record comment activity if comment is entered - content foreach { - (if(issue.isPullRequest) recordCommentPullRequestActivity _ else recordCommentIssueActivity _) - (owner, name, userName, issue.issueId, _) - } - recordActivity foreach ( _ (owner, name, userName, issue.issueId, issue.title) ) - - // extract references and create refer comment - content.map { content => - createReferComment(owner, name, issue, content, context.loginAccount.get) - } - - // call web hooks - action match { - case None => commentId.map { commentIdSome => callIssueCommentWebHook(repository, issue, commentIdSome, context.loginAccount.get) } - case Some(act) => { - val webHookAction = act match { - case "open" => "opened" - case "reopen" => "reopened" - case "close" => "closed" - case _ => act + val (action, recordActivity) = actionOpt + .collect { + case "close" if(!issue.closed) => true -> + (Some("close") -> Some(if(issue.isPullRequest) recordClosePullRequestActivity _ else recordCloseIssueActivity _)) + case "reopen" if(issue.closed) => false -> + (Some("reopen") -> Some(recordReopenIssueActivity _)) } - if (issue.isPullRequest) { - callPullRequestWebHook(webHookAction, repository, issue.issueId, context.baseUrl, context.loginAccount.get) - } else { - callIssuesWebHook(webHookAction, repository, issue, context.baseUrl, context.loginAccount.get) + .map { case (closed, t) => + updateClosed(owner, name, issue.issueId, closed) + t } - } - } + .getOrElse(None -> None) - // notifications - Notifier() match { - case f => - content foreach { - f.toNotify(repository, issue, _){ - Notifier.msgComment(s"${context.baseUrl}/${owner}/${name}/${ - if(issue.isPullRequest) "pull" else "issues"}/${issue.issueId}#comment-${commentId.get}") + val commentId = (content, action) match { + case (None, None) => None + case (None, Some(action)) => Some(createComment(owner, name, userName, issue.issueId, action.capitalize, action)) + case (Some(content), _) => Some(createComment(owner, name, userName, issue.issueId, content, action.map(_+ "_comment").getOrElse("comment"))) + } + + // record comment activity if comment is entered + content foreach { + (if(issue.isPullRequest) recordCommentPullRequestActivity _ else recordCommentIssueActivity _) + (owner, name, userName, issue.issueId, _) + } + recordActivity foreach ( _ (owner, name, userName, issue.issueId, issue.title) ) + + // extract references and create refer comment + content.map { content => + createReferComment(owner, name, issue, content, loginAccount) + } + + // call web hooks + action match { + case None => commentId.map { commentIdSome => callIssueCommentWebHook(repository, issue, commentIdSome, loginAccount) } + case Some(act) => { + val webHookAction = act match { + case "open" => "opened" + case "reopen" => "reopened" + case "close" => "closed" + case _ => act + } + if (issue.isPullRequest) { + callPullRequestWebHook(webHookAction, repository, issue.issueId, context.baseUrl, loginAccount) + } else { + callIssuesWebHook(webHookAction, repository, issue, context.baseUrl, loginAccount) } } - action foreach { - f.toNotify(repository, issue, _){ - Notifier.msgStatus(s"${context.baseUrl}/${owner}/${name}/issues/${issue.issueId}") - } - } - } + } - commentId.map( issue -> _ ) + // notifications + Notifier() match { + case f => + content foreach { + f.toNotify(repository, issue, _){ + Notifier.msgComment(s"${context.baseUrl}/${owner}/${name}/${ + if(issue.isPullRequest) "pull" else "issues"}/${issue.issueId}#comment-${commentId.get}") + } + } + action foreach { + f.toNotify(repository, issue, _){ + Notifier.msgStatus(s"${context.baseUrl}/${owner}/${name}/issues/${issue.issueId}") + } + } + } + + commentId.map( issue -> _ ) + } } } diff --git a/src/main/scala/gitbucket/core/service/IssueCreationService.scala b/src/main/scala/gitbucket/core/service/IssueCreationService.scala new file mode 100644 index 0000000..a18dad3 --- /dev/null +++ b/src/main/scala/gitbucket/core/service/IssueCreationService.scala @@ -0,0 +1,74 @@ +package gitbucket.core.service + +import gitbucket.core.controller.Context +import gitbucket.core.model.{Account, Issue} +import gitbucket.core.model.Profile.profile.blockingApi._ +import gitbucket.core.service.RepositoryService.RepositoryInfo +import gitbucket.core.util.Notifier +import gitbucket.core.util.Implicits._ + +// TODO: Merged with IssuesService? +trait IssueCreationService { + + self: RepositoryService with WebHookIssueCommentService with LabelsService with IssuesService with ActivityService => + + def createIssue(repository: RepositoryInfo, title:String, body:Option[String], + assignee: Option[String], milestoneId: Option[Int], labelNames: Seq[String], + loginAccount: Account)(implicit context: Context, s: Session) : Issue = { + + val owner = repository.owner + val name = repository.name + val userName = loginAccount.userName + val manageable = isIssueManageable(repository) + + // insert issue + val issueId = insertIssue(owner, name, userName, title, body, + if (manageable) assignee else None, + if (manageable) milestoneId else None) + val issue: Issue = getIssue(owner, name, issueId.toString).get + + // insert labels + if (manageable) { + val labels = getLabels(owner, name) + labelNames.map { labelName => + labels.find(_.labelName == labelName).map { label => + registerIssueLabel(owner, name, issueId, label.labelId) + } + } + } + + // record activity + recordCreateIssueActivity(owner, name, userName, issueId, title) + + // extract references and create refer comment + createReferComment(owner, name, issue, title + " " + body.getOrElse(""), loginAccount) + + // call web hooks + callIssuesWebHook("opened", repository, issue, context.baseUrl, loginAccount) + + // notifications + Notifier().toNotify(repository, issue, body.getOrElse("")) { + Notifier.msgIssue(s"${context.baseUrl}/${owner}/${name}/issues/${issueId}") + } + issue + } + + /** + * Tests whether an logged-in user can manage issues. + */ + protected def isIssueManageable(repository: RepositoryInfo)(implicit context: Context, s: Session): Boolean = { + hasDeveloperRole(repository.owner, repository.name, context.loginAccount) + } + + /** + * Tests whether an logged-in user can post issues. + */ + protected def isIssueEditable(repository: RepositoryInfo)(implicit context: Context, s: Session): Boolean = { + repository.repository.options.issuesOption match { + case "ALL" => !repository.repository.isPrivate && context.loginAccount.isDefined + case "PUBLIC" => hasGuestRole(repository.owner, repository.name, context.loginAccount) + case "PRIVATE" => hasDeveloperRole(repository.owner, repository.name, context.loginAccount) + case "DISABLE" => false + } + } +} diff --git a/src/main/scala/gitbucket/core/service/IssuesService.scala b/src/main/scala/gitbucket/core/service/IssuesService.scala index 97bbc6c..5543e9b 100644 --- a/src/main/scala/gitbucket/core/service/IssuesService.scala +++ b/src/main/scala/gitbucket/core/service/IssuesService.scala @@ -21,7 +21,7 @@ else None def getComments(owner: String, repository: String, issueId: Int)(implicit s: Session) = - IssueComments filter (_.byIssue(owner, repository, issueId)) list + IssueComments filter (_.byIssue(owner, repository, issueId)) sortBy(_.commentId asc) list /** @return IssueComment and commentedUser and Issue */ def getCommentsForApi(owner: String, repository: String, issueId: Int)(implicit s: Session): List[(IssueComment, Account, Issue)] = @@ -149,7 +149,19 @@ } /** for api - * + * @return (issue, issueUser, commentCount) + */ + def searchIssueByApi(condition: IssueSearchCondition, offset: Int, limit: Int, repos: (String, String)*) + (implicit s: Session): List[(Issue, Account)] = { + // get issues and comment count and labels + searchIssueQueryBase(condition, false, offset, limit, repos) + .join(Accounts).on { case (((t1, t2), i), t3) => t3.userName === t1.openedUserName } + .sortBy { case (((t1, t2), i), t3) => i asc } + .map { case (((t1, t2), i), t3) => (t1, t3) } + .list + } + + /** for api * @return (issue, issueUser, commentCount, pullRequest, headRepo, headOwner) */ def searchPullRequestByApi(condition: IssueSearchCondition, offset: Int, limit: Int, repos: (String, String)*) @@ -232,7 +244,7 @@ } exists), condition.mentioned.isDefined) } - def createIssue(owner: String, repository: String, loginUser: String, title: String, content: Option[String], + def insertIssue(owner: String, repository: String, loginUser: String, title: String, content: Option[String], assignedUserName: Option[String], milestoneId: Option[Int], isPullRequest: Boolean = false)(implicit s: Session) = // next id number diff --git a/src/main/scala/gitbucket/core/service/MergeService.scala b/src/main/scala/gitbucket/core/service/MergeService.scala index 74a329d..d58a4ec 100644 --- a/src/main/scala/gitbucket/core/service/MergeService.scala +++ b/src/main/scala/gitbucket/core/service/MergeService.scala @@ -195,4 +195,4 @@ private def parseCommit(id:ObjectId) = using(new RevWalk( repository ))(_.parseCommit(id)) } -} \ No newline at end of file +} diff --git a/src/main/scala/gitbucket/core/service/RepositoryService.scala b/src/main/scala/gitbucket/core/service/RepositoryService.scala index dfb0f71..79f0852 100644 --- a/src/main/scala/gitbucket/core/service/RepositoryService.scala +++ b/src/main/scala/gitbucket/core/service/RepositoryService.scala @@ -1,12 +1,15 @@ package gitbucket.core.service import gitbucket.core.controller.Context -import gitbucket.core.util.JGitUtil -import gitbucket.core.model.{Collaborator, Repository, RepositoryOptions, Account, Role} +import gitbucket.core.util._ +import gitbucket.core.util.ControlUtil._ +import gitbucket.core.model.{Account, Collaborator, Repository, RepositoryOptions, Role} import gitbucket.core.model.Profile._ import gitbucket.core.model.Profile.profile._ import gitbucket.core.model.Profile.profile.blockingApi._ import gitbucket.core.model.Profile.dateColumnType +import gitbucket.core.util.JGitUtil.FileInfo +import org.eclipse.jgit.api.Git trait RepositoryService { self: AccountService => import RepositoryService._ @@ -423,6 +426,36 @@ } .sortBy(_.userName asc).map(t => t.userName -> t.repositoryName).list + private val templateExtensions = Seq("md", "markdown") + + /** + * Returns content of template set per repository. + * + * @param repository the repository information + * @param fileBaseName the file basename without extension of template + * @return The content of template if the repository has it, otherwise empty string. + */ + def getContentTemplate(repository: RepositoryInfo, fileBaseName: String)(implicit s: Session): String = { + val withExtFilenames = templateExtensions.map(extension => s"${fileBaseName.toLowerCase()}.${extension}") + + def choiceTemplate(files: List[FileInfo]): Option[FileInfo] = + files.find { f => + f.name.toLowerCase() == fileBaseName + }.orElse { + files.find(f => withExtFilenames.contains(f.name.toLowerCase())) + } + + // Get template file from project root. When didn't find, will lookup default folder. + using(Git.open(Directory.getRepositoryDir(repository.owner, repository.name))) { git => + choiceTemplate(JGitUtil.getFileList(git, repository.repository.defaultBranch, ".")).orElse { + choiceTemplate(JGitUtil.getFileList(git, repository.repository.defaultBranch, ".gitbucket")) + }.map { file => + JGitUtil.getContentFromId(git, file.id, true).collect { + case bytes if FileUtil.isText(bytes) => StringUtil.convertFromByteArray(bytes) + } + } getOrElse None + } getOrElse "" + } } object RepositoryService { diff --git a/src/main/scala/gitbucket/core/service/SystemSettingsService.scala b/src/main/scala/gitbucket/core/service/SystemSettingsService.scala index c2aa28d..0e1b677 100644 --- a/src/main/scala/gitbucket/core/service/SystemSettingsService.scala +++ b/src/main/scala/gitbucket/core/service/SystemSettingsService.scala @@ -32,6 +32,7 @@ smtp.user.foreach(props.setProperty(SmtpUser, _)) smtp.password.foreach(props.setProperty(SmtpPassword, _)) smtp.ssl.foreach(x => props.setProperty(SmtpSsl, x.toString)) + smtp.starttls.foreach(x => props.setProperty(SmtpStarttls, x.toString)) smtp.fromAddress.foreach(props.setProperty(SmtpFromAddress, _)) smtp.fromName.foreach(props.setProperty(SmtpFromName, _)) } @@ -87,6 +88,7 @@ getOptionValue(props, SmtpUser, None), getOptionValue(props, SmtpPassword, None), getOptionValue[Boolean](props, SmtpSsl, None), + getOptionValue[Boolean](props, SmtpStarttls, None), getOptionValue(props, SmtpFromAddress, None), getOptionValue(props, SmtpFromName, None))) } else { @@ -168,6 +170,7 @@ user: Option[String], password: Option[String], ssl: Option[Boolean], + starttls: Option[Boolean], fromAddress: Option[String], fromName: Option[String]) @@ -176,6 +179,9 @@ port:Int, genericUser:String) + case class Lfs( + serverUrl: Option[String]) + val DefaultSshPort = 29418 val DefaultSmtpPort = 25 val DefaultLdapPort = 389 @@ -197,6 +203,7 @@ private val SmtpUser = "smtp.user" private val SmtpPassword = "smtp.password" private val SmtpSsl = "smtp.ssl" + private val SmtpStarttls = "smtp.starttls" private val SmtpFromAddress = "smtp.from_address" private val SmtpFromName = "smtp.from_name" private val LdapAuthentication = "ldap_authentication" diff --git a/src/main/scala/gitbucket/core/service/WebHookService.scala b/src/main/scala/gitbucket/core/service/WebHookService.scala index 1083868..b83257a 100644 --- a/src/main/scala/gitbucket/core/service/WebHookService.scala +++ b/src/main/scala/gitbucket/core/service/WebHookService.scala @@ -1,7 +1,5 @@ package gitbucket.core.service -import java.util.Date - import fr.brouillard.oss.security.xhub.XHub import fr.brouillard.oss.security.xhub.XHub.{XHubConverter, XHubDigest} import gitbucket.core.api._ diff --git a/src/main/scala/gitbucket/core/servlet/ApiAuthenticationFilter.scala b/src/main/scala/gitbucket/core/servlet/ApiAuthenticationFilter.scala index 871c292..c3bd85c 100644 --- a/src/main/scala/gitbucket/core/servlet/ApiAuthenticationFilter.scala +++ b/src/main/scala/gitbucket/core/servlet/ApiAuthenticationFilter.scala @@ -7,8 +7,6 @@ import gitbucket.core.service.SystemSettingsService.SystemSettings import gitbucket.core.service.{AccessTokenService, AccountService, SystemSettingsService} import gitbucket.core.util.{AuthUtil, Keys} -import org.scalatra.servlet.ServletApiImplicits._ -import org.scalatra._ class ApiAuthenticationFilter extends Filter with AccessTokenService with AccountService with SystemSettingsService { diff --git a/src/main/scala/gitbucket/core/servlet/GitAuthenticationFilter.scala b/src/main/scala/gitbucket/core/servlet/GitAuthenticationFilter.scala index d0b92bd..d474488 100644 --- a/src/main/scala/gitbucket/core/servlet/GitAuthenticationFilter.scala +++ b/src/main/scala/gitbucket/core/servlet/GitAuthenticationFilter.scala @@ -6,6 +6,7 @@ import gitbucket.core.service.SystemSettingsService.SystemSettings import gitbucket.core.service.{RepositoryService, AccountService, SystemSettingsService} import gitbucket.core.util.{Keys, Implicits, AuthUtil} +import gitbucket.core.model.Profile.profile._ import org.slf4j.LoggerFactory import Implicits._ @@ -70,42 +71,47 @@ private def defaultRepository(request: HttpServletRequest, response: HttpServletResponse, chain: FilterChain, settings: SystemSettings, isUpdating: Boolean): Unit = { - implicit val r = request - - request.paths match { + val action = request.paths match { case Array(_, repositoryOwner, repositoryName, _*) => - getRepository(repositoryOwner, repositoryName.replaceFirst("\\.wiki\\.git$|\\.git$", "")) match { - case Some(repository) => { - if(!isUpdating && !repository.repository.isPrivate && settings.allowAnonymousAccess){ - chain.doFilter(request, response) - } else { - val passed = for { - auth <- Option(request.getHeader("Authorization")) - Array(username, password) = AuthUtil.decodeAuthHeader(auth).split(":", 2) - account <- authenticate(settings, username, password) - } yield if(isUpdating || repository.repository.isPrivate){ - if(hasDeveloperRole(repository.owner, repository.name, Some(account))){ + Database() withSession { implicit session => + getRepository(repositoryOwner, repositoryName.replaceFirst("\\.wiki\\.git$|\\.git$", "")) match { + case Some(repository) => { + val execute = if (!isUpdating && !repository.repository.isPrivate && settings.allowAnonymousAccess) { + // Authentication is not required + true + } else { + // Authentication is required + val passed = for { + auth <- Option(request.getHeader("Authorization")) + Array(username, password) = AuthUtil.decodeAuthHeader(auth).split(":", 2) + account <- authenticate(settings, username, password) + } yield if (isUpdating || repository.repository.isPrivate) { + if (hasDeveloperRole(repository.owner, repository.name, Some(account))) { request.setAttribute(Keys.Request.UserName, account.userName) true } else false } else true + passed.getOrElse(false) + } - if(passed.getOrElse(false)){ - chain.doFilter(request, response) + if (execute) { + () => chain.doFilter(request, response) } else { - AuthUtil.requireAuth(response) + () => AuthUtil.requireAuth(response) } } - } - case None => { - logger.debug(s"Repository ${repositoryOwner}/${repositoryName} is not found.") - response.sendError(HttpServletResponse.SC_NOT_FOUND) + case None => () => { + logger.debug(s"Repository ${repositoryOwner}/${repositoryName} is not found.") + response.sendError(HttpServletResponse.SC_NOT_FOUND) + } } } - case _ => { + case _ => () => { logger.debug(s"Not enough path arguments: ${request.paths}") response.sendError(HttpServletResponse.SC_NOT_FOUND) } } + + action() } } \ No newline at end of file diff --git a/src/main/scala/gitbucket/core/servlet/GitLfsTransferServlet.scala b/src/main/scala/gitbucket/core/servlet/GitLfsTransferServlet.scala new file mode 100644 index 0000000..ed14f30 --- /dev/null +++ b/src/main/scala/gitbucket/core/servlet/GitLfsTransferServlet.scala @@ -0,0 +1,83 @@ +package gitbucket.core.servlet + +import java.io.{File, FileInputStream, FileOutputStream} +import java.text.MessageFormat +import javax.servlet.http.{HttpServlet, HttpServletRequest, HttpServletResponse} + +import gitbucket.core.util.{FileUtil, StringUtil} +import org.apache.commons.io.{FileUtils, IOUtils} +import org.json4s.jackson.Serialization._ +import org.apache.http.HttpStatus +import gitbucket.core.util.ControlUtil._ + +/** + * Provides GitLFS Transfer API + * https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md + */ +class GitLfsTransferServlet extends HttpServlet { + + private implicit val jsonFormats = gitbucket.core.api.JsonFormat.jsonFormats + private val LongObjectIdLength = 32 + private val LongObjectIdStringLength = LongObjectIdLength * 2 + + override protected def doGet(req: HttpServletRequest, res: HttpServletResponse): Unit = { + for { + (owner, repository, oid) <- getPathInfo(req, res) if checkToken(req, oid) + } yield { + val file = new File(FileUtil.getLfsFilePath(owner, repository, oid)) + if(file.exists()){ + res.setStatus(HttpStatus.SC_OK) + res.setContentType("application/octet-stream") + res.setContentLength(file.length.toInt) + using(new FileInputStream(file), res.getOutputStream){ (in, out) => + IOUtils.copy(in, out) + out.flush() + } + } else { + sendError(res, HttpStatus.SC_NOT_FOUND, + MessageFormat.format("Object ''{0}'' not found", oid)) + } + } + } + + override protected def doPut(req: HttpServletRequest, res: HttpServletResponse): Unit = { + for { + (owner, repository, oid) <- getPathInfo(req, res) if checkToken(req, oid) + } yield { + val file = new File(FileUtil.getLfsFilePath(owner, repository, oid)) + FileUtils.forceMkdir(file.getParentFile) + using(req.getInputStream, new FileOutputStream(file)){ (in, out) => + IOUtils.copy(in, out) + } + res.setStatus(HttpStatus.SC_OK) + } + } + + private def checkToken(req: HttpServletRequest, oid: String): Boolean = { + val token = req.getHeader("Authorization") + if(token != null){ + val Array(expireAt, targetOid) = StringUtil.decodeBlowfish(token).split(" ") + oid == targetOid && expireAt.toLong > System.currentTimeMillis + } else { + false + } + } + + private def getPathInfo(req: HttpServletRequest, res: HttpServletResponse): Option[(String, String, String)] = { + req.getRequestURI.substring(1).split("/").reverse match { + case Array(oid, repository, owner, _*) => Some((owner, repository, oid)) + case _ => None + } + } + + private def sendError(res: HttpServletResponse, status: Int, message: String): Unit = { + res.setStatus(status) + using(res.getWriter()){ out => + out.write(write(GitLfs.Error(message))) + out.flush() + } + } + +} + + diff --git a/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala b/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala index e85804e..8adb1cf 100644 --- a/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala +++ b/src/main/scala/gitbucket/core/servlet/GitRepositoryServlet.scala @@ -1,9 +1,11 @@ package gitbucket.core.servlet import java.io.File +import java.util.Date import gitbucket.core.api -import gitbucket.core.model.{Session, WebHook} +import gitbucket.core.model.WebHook +import gitbucket.core.model.Profile.profile._ import gitbucket.core.plugin.{GitRepositoryRouting, PluginRegistry} import gitbucket.core.service.IssuesService.IssueSearchCondition import gitbucket.core.service.WebHookService._ @@ -11,16 +13,16 @@ import gitbucket.core.util.ControlUtil._ import gitbucket.core.util.Implicits._ import gitbucket.core.util._ - import org.eclipse.jgit.api.Git import org.eclipse.jgit.http.server.GitServlet import org.eclipse.jgit.lib._ import org.eclipse.jgit.transport._ import org.eclipse.jgit.transport.resolver._ import org.slf4j.LoggerFactory - import javax.servlet.ServletConfig -import javax.servlet.http.{HttpServletResponse, HttpServletRequest} +import javax.servlet.http.{HttpServletRequest, HttpServletResponse} + +import org.json4s.jackson.Serialization._ /** @@ -32,7 +34,8 @@ class GitRepositoryServlet extends GitServlet with SystemSettingsService { private val logger = LoggerFactory.getLogger(classOf[GitRepositoryServlet]) - + private implicit val jsonFormats = gitbucket.core.api.JsonFormat.jsonFormats + override def init(config: ServletConfig): Unit = { setReceivePackFactory(new GitBucketReceivePackFactory()) @@ -45,15 +48,75 @@ override def service(req: HttpServletRequest, res: HttpServletResponse): Unit = { val agent = req.getHeader("USER-AGENT") val index = req.getRequestURI.indexOf(".git") - if(index >= 0 && (agent == null || agent.toLowerCase.indexOf("git/") < 0)){ + if(index >= 0 && (agent == null || agent.toLowerCase.indexOf("git") < 0)){ // redirect for browsers val paths = req.getRequestURI.substring(0, index).split("/") res.sendRedirect(baseUrl(req) + "/" + paths.dropRight(1).last + "/" + paths.last) + + } else if(req.getMethod.toUpperCase == "POST" && req.getRequestURI.endsWith("/info/lfs/objects/batch")){ + serviceGitLfsBatchAPI(req, res) + } else { // response for git client super.service(req, res) } } + + /** + * Provides GitLFS Batch API + * https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md + */ + protected def serviceGitLfsBatchAPI(req: HttpServletRequest, res: HttpServletResponse): Unit = { + val batchRequest = read[GitLfs.BatchRequest](req.getInputStream) + val settings = loadSystemSettings() + + settings.baseUrl match { + case None => { + throw new IllegalStateException("lfs.server_url is not configured.") + } + case Some(baseUrl) => { + val index = req.getRequestURI.indexOf(".git") + if(index >= 0){ + req.getRequestURI.substring(0, index).split("/").reverse match { + case Array(repository, owner, _*) => + val timeout = System.currentTimeMillis + (60000 * 10) // 10 min. + val batchResponse = batchRequest.operation match { + case "upload" => + GitLfs.BatchUploadResponse("basic", batchRequest.objects.map { requestObject => + GitLfs.BatchResponseObject(requestObject.oid, requestObject.size, true, + GitLfs.Actions( + upload = Some(GitLfs.Action( + href = baseUrl + "/git-lfs/" + owner + "/" + repository + "/" + requestObject.oid, + header = Map("Authorization" -> StringUtil.encodeBlowfish(timeout + " " + requestObject.oid)), + expires_at = new Date(timeout) + )) + ) + ) + }) + case "download" => + GitLfs.BatchUploadResponse("basic", batchRequest.objects.map { requestObject => + GitLfs.BatchResponseObject(requestObject.oid, requestObject.size, true, + GitLfs.Actions( + download = Some(GitLfs.Action( + href = baseUrl + "/git-lfs/" + owner + "/" + repository + "/" + requestObject.oid, + header = Map("Authorization" -> StringUtil.encodeBlowfish(timeout + " " + requestObject.oid)), + expires_at = new Date(timeout) + )) + ) + ) + }) + } + + res.setContentType("application/vnd.git-lfs+json") + using(res.getWriter){ out => + out.print(write(batchResponse)) + out.flush() + } + } + } + } + } + } } class GitBucketRepositoryResolver(parent: FileResolver[HttpServletRequest]) extends RepositoryResolver[HttpServletRequest] { @@ -107,7 +170,7 @@ import scala.collection.JavaConverters._ -class CommitLogHook(owner: String, repository: String, pusher: String, baseUrl: String)(implicit session: Session) +class CommitLogHook(owner: String, repository: String, pusher: String, baseUrl: String)/*(implicit session: Session)*/ extends PostReceiveHook with PreReceiveHook with RepositoryService with AccountService with IssuesService with ActivityService with PullRequestService with WebHookService with WebHookPullRequestService with CommitsService { @@ -116,119 +179,165 @@ private var existIds: Seq[String] = Nil def onPreReceive(receivePack: ReceivePack, commands: java.util.Collection[ReceiveCommand]): Unit = { - try { - commands.asScala.foreach { command => - // call pre-commit hook - PluginRegistry().getReceiveHooks - .flatMap(_.preReceive(owner, repository, receivePack, command, pusher)) - .headOption.foreach { error => - command.setResult(ReceiveCommand.Result.REJECTED_OTHER_REASON, error) + Database() withTransaction { implicit session => + try { + commands.asScala.foreach { command => + // call pre-commit hook + PluginRegistry().getReceiveHooks + .flatMap(_.preReceive(owner, repository, receivePack, command, pusher)) + .headOption.foreach { error => + command.setResult(ReceiveCommand.Result.REJECTED_OTHER_REASON, error) + } } - } - using(Git.open(Directory.getRepositoryDir(owner, repository))) { git => - existIds = JGitUtil.getAllCommitIds(git) - } - } catch { - case ex: Exception => { - logger.error(ex.toString, ex) - throw ex + using(Git.open(Directory.getRepositoryDir(owner, repository))) { git => + existIds = JGitUtil.getAllCommitIds(git) + } + } catch { + case ex: Exception => { + logger.error(ex.toString, ex) + throw ex + } } } } def onPostReceive(receivePack: ReceivePack, commands: java.util.Collection[ReceiveCommand]): Unit = { - try { - using(Git.open(Directory.getRepositoryDir(owner, repository))) { git => - JGitUtil.removeCache(git) + Database() withTransaction { implicit session => + try { + using(Git.open(Directory.getRepositoryDir(owner, repository))) { git => + JGitUtil.removeCache(git) - val pushedIds = scala.collection.mutable.Set[String]() - commands.asScala.foreach { command => - logger.debug(s"commandType: ${command.getType}, refName: ${command.getRefName}") - implicit val apiContext = api.JsonFormat.Context(baseUrl) - val refName = command.getRefName.split("/") - val branchName = refName.drop(2).mkString("/") - val commits = if (refName(1) == "tags") { - Nil - } else { - command.getType match { - case ReceiveCommand.Type.DELETE => Nil - case _ => JGitUtil.getCommitLog(git, command.getOldId.name, command.getNewId.name) - } - } - - // Retrieve all issue count in the repository - val issueCount = - countIssue(IssueSearchCondition(state = "open"), false, owner -> repository) + - countIssue(IssueSearchCondition(state = "closed"), false, owner -> repository) - - val repositoryInfo = getRepository(owner, repository).get - - // Extract new commit and apply issue comment - val defaultBranch = repositoryInfo.repository.defaultBranch - val newCommits = commits.flatMap { commit => - if (!existIds.contains(commit.id) && !pushedIds.contains(commit.id)) { - if (issueCount > 0) { - pushedIds.add(commit.id) - createIssueComment(owner, repository, commit) - // close issues - if(refName(1) == "heads" && branchName == defaultBranch && command.getType == ReceiveCommand.Type.UPDATE){ - closeIssuesFromMessage(commit.fullMessage, pusher, owner, repository) - } + val pushedIds = scala.collection.mutable.Set[String]() + commands.asScala.foreach { command => + logger.debug(s"commandType: ${command.getType}, refName: ${command.getRefName}") + implicit val apiContext = api.JsonFormat.Context(baseUrl) + val refName = command.getRefName.split("/") + val branchName = refName.drop(2).mkString("/") + val commits = if (refName(1) == "tags") { + Nil + } else { + command.getType match { + case ReceiveCommand.Type.DELETE => Nil + case _ => JGitUtil.getCommitLog(git, command.getOldId.name, command.getNewId.name) } - Some(commit) - } else None - } - - // record activity - if(refName(1) == "heads"){ - command.getType match { - case ReceiveCommand.Type.CREATE => recordCreateBranchActivity(owner, repository, pusher, branchName) - case ReceiveCommand.Type.UPDATE => recordPushActivity(owner, repository, pusher, branchName, newCommits) - case ReceiveCommand.Type.DELETE => recordDeleteBranchActivity(owner, repository, pusher, branchName) - case _ => } - } else if(refName(1) == "tags"){ - command.getType match { - case ReceiveCommand.Type.CREATE => recordCreateTagActivity(owner, repository, pusher, branchName, newCommits) - case ReceiveCommand.Type.DELETE => recordDeleteTagActivity(owner, repository, pusher, branchName, newCommits) - case _ => - } - } - if(refName(1) == "heads"){ - command.getType match { - case ReceiveCommand.Type.CREATE | - ReceiveCommand.Type.UPDATE | - ReceiveCommand.Type.UPDATE_NONFASTFORWARD => - updatePullRequests(owner, repository, branchName) - getAccountByUserName(pusher).map{ pusherAccount => - callPullRequestWebHookByRequestBranch("synchronize", repositoryInfo, branchName, baseUrl, pusherAccount) + // Retrieve all issue count in the repository + val issueCount = + countIssue(IssueSearchCondition(state = "open"), false, owner -> repository) + + countIssue(IssueSearchCondition(state = "closed"), false, owner -> repository) + + val repositoryInfo = getRepository(owner, repository).get + + // Extract new commit and apply issue comment + val defaultBranch = repositoryInfo.repository.defaultBranch + val newCommits = commits.flatMap { commit => + if (!existIds.contains(commit.id) && !pushedIds.contains(commit.id)) { + if (issueCount > 0) { + pushedIds.add(commit.id) + createIssueComment(owner, repository, commit) + // close issues + if (refName(1) == "heads" && branchName == defaultBranch && command.getType == ReceiveCommand.Type.UPDATE) { + closeIssuesFromMessage(commit.fullMessage, pusher, owner, repository) + } } - case _ => + Some(commit) + } else None } - } - // call web hook - callWebHookOf(owner, repository, WebHook.Push){ - for(pusherAccount <- getAccountByUserName(pusher); - ownerAccount <- getAccountByUserName(owner)) yield { - WebHookPushPayload(git, pusherAccount, command.getRefName, repositoryInfo, newCommits, ownerAccount, - newId = command.getNewId(), oldId = command.getOldId()) + // record activity + if (refName(1) == "heads") { + command.getType match { + case ReceiveCommand.Type.CREATE => recordCreateBranchActivity(owner, repository, pusher, branchName) + case ReceiveCommand.Type.UPDATE => recordPushActivity(owner, repository, pusher, branchName, newCommits) + case ReceiveCommand.Type.DELETE => recordDeleteBranchActivity(owner, repository, pusher, branchName) + case _ => + } + } else if (refName(1) == "tags") { + command.getType match { + case ReceiveCommand.Type.CREATE => recordCreateTagActivity(owner, repository, pusher, branchName, newCommits) + case ReceiveCommand.Type.DELETE => recordDeleteTagActivity(owner, repository, pusher, branchName, newCommits) + case _ => + } } - } - // call post-commit hook - PluginRegistry().getReceiveHooks.foreach(_.postReceive(owner, repository, receivePack, command, pusher)) + if (refName(1) == "heads") { + command.getType match { + case ReceiveCommand.Type.CREATE | + ReceiveCommand.Type.UPDATE | + ReceiveCommand.Type.UPDATE_NONFASTFORWARD => + updatePullRequests(owner, repository, branchName) + getAccountByUserName(pusher).map { pusherAccount => + callPullRequestWebHookByRequestBranch("synchronize", repositoryInfo, branchName, baseUrl, pusherAccount) + } + case _ => + } + } + + // call web hook + callWebHookOf(owner, repository, WebHook.Push) { + for (pusherAccount <- getAccountByUserName(pusher); + ownerAccount <- getAccountByUserName(owner)) yield { + WebHookPushPayload(git, pusherAccount, command.getRefName, repositoryInfo, newCommits, ownerAccount, + newId = command.getNewId(), oldId = command.getOldId()) + } + } + + // call post-commit hook + PluginRegistry().getReceiveHooks.foreach(_.postReceive(owner, repository, receivePack, command, pusher)) + } } - } - // update repository last modified time. - updateLastActivityDate(owner, repository) - } catch { - case ex: Exception => { - logger.error(ex.toString, ex) - throw ex + // update repository last modified time. + updateLastActivityDate(owner, repository) + } catch { + case ex: Exception => { + logger.error(ex.toString, ex) + throw ex + } } } } } + +object GitLfs { + + case class BatchRequest( + operation: String, + transfers: Seq[String], + objects: Seq[BatchRequestObject] + ) + + case class BatchRequestObject( + oid: String, + size: Long + ) + + case class BatchUploadResponse( + transfer: String, + objects: Seq[BatchResponseObject] + ) + + case class BatchResponseObject( + oid: String, + size: Long, + authenticated: Boolean, + actions: Actions + ) + + case class Actions( + download: Option[Action] = None, + upload: Option[Action] = None + ) + + case class Action( + href: String, + header: Map[String, String] = Map.empty, + expires_at: Date + ) + + case class Error( + message: String + ) + +} diff --git a/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala b/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala index 891f1f0..f77d3ee 100644 --- a/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala +++ b/src/main/scala/gitbucket/core/servlet/TransactionFilter.scala @@ -22,8 +22,9 @@ def destroy(): Unit = {} def doFilter(req: ServletRequest, res: ServletResponse, chain: FilterChain): Unit = { - if(req.asInstanceOf[HttpServletRequest].getServletPath().startsWith("/assets/")){ - // assets don't need transaction + val servletPath = req.asInstanceOf[HttpServletRequest].getServletPath() + if(servletPath.startsWith("/assets/") || servletPath == "/git" || servletPath == "/git-lfs"){ + // assets and git-lfs don't need transaction chain.doFilter(req, res) } else { Database() withTransaction { session => diff --git a/src/main/scala/gitbucket/core/ssh/GitCommand.scala b/src/main/scala/gitbucket/core/ssh/GitCommand.scala index 8897eb6..a28a150 100644 --- a/src/main/scala/gitbucket/core/ssh/GitCommand.scala +++ b/src/main/scala/gitbucket/core/ssh/GitCommand.scala @@ -31,24 +31,22 @@ @volatile protected var callback: ExitCallback = null @volatile private var authUser:Option[String] = None - protected def runTask(authUser: String)(implicit session: Session): Unit + protected def runTask(authUser: String): Unit private def newTask(): Runnable = new Runnable { override def run(): Unit = { authUser match { case Some(authUser) => - Database() withTransaction { implicit session => - try { - runTask(authUser) - callback.onExit(0) - } catch { - case e: RepositoryNotFoundException => - logger.info(e.getMessage) - callback.onExit(1, "Repository Not Found") - case e: Throwable => - logger.error(e.getMessage, e) - callback.onExit(1) - } + try { + runTask(authUser) + callback.onExit(0) + } catch { + case e: RepositoryNotFoundException => + logger.info(e.getMessage) + callback.onExit(1, "Repository Not Found") + case e: Throwable => + logger.error(e.getMessage, e) + callback.onExit(1) } case None => val message = "User not authenticated" @@ -103,14 +101,18 @@ class DefaultGitUploadPack(owner: String, repoName: String) extends DefaultGitCommand(owner, repoName) with RepositoryService with AccountService { - override protected def runTask(user: String)(implicit session: Session): Unit = { - getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", "")).foreach { repositoryInfo => - if(!repositoryInfo.repository.isPrivate || isWritableUser(user, repositoryInfo)){ - using(Git.open(getRepositoryDir(owner, repoName))) { git => - val repository = git.getRepository - val upload = new UploadPack(repository) - upload.upload(in, out, err) - } + override protected def runTask(user: String): Unit = { + val execute = Database() withSession { implicit session => + getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", "")).map { repositoryInfo => + !repositoryInfo.repository.isPrivate || isWritableUser(user, repositoryInfo) + }.getOrElse(false) + } + + if(execute){ + using(Git.open(getRepositoryDir(owner, repoName))) { git => + val repository = git.getRepository + val upload = new UploadPack(repository) + upload.upload(in, out, err) } } } @@ -119,19 +121,23 @@ class DefaultGitReceivePack(owner: String, repoName: String, baseUrl: String) extends DefaultGitCommand(owner, repoName) with RepositoryService with AccountService { - override protected def runTask(user: String)(implicit session: Session): Unit = { - getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", "")).foreach { repositoryInfo => - if(isWritableUser(user, repositoryInfo)){ - using(Git.open(getRepositoryDir(owner, repoName))) { git => - val repository = git.getRepository - val receive = new ReceivePack(repository) - if(!repoName.endsWith(".wiki")){ - val hook = new CommitLogHook(owner, repoName, user, baseUrl) - receive.setPreReceiveHook(hook) - receive.setPostReceiveHook(hook) - } - receive.receive(in, out, err) + override protected def runTask(user: String): Unit = { + val execute = Database() withSession { implicit session => + getRepository(owner, repoName.replaceFirst("\\.wiki\\Z", "")).map { repositoryInfo => + isWritableUser(user, repositoryInfo) + }.getOrElse(false) + } + + if(execute) { + using(Git.open(getRepositoryDir(owner, repoName))) { git => + val repository = git.getRepository + val receive = new ReceivePack(repository) + if (!repoName.endsWith(".wiki")) { + val hook = new CommitLogHook(owner, repoName, user, baseUrl) + receive.setPreReceiveHook(hook) + receive.setPostReceiveHook(hook) } + receive.receive(in, out, err) } } } @@ -140,8 +146,12 @@ class PluginGitUploadPack(repoName: String, routing: GitRepositoryRouting) extends GitCommand with SystemSettingsService { - override protected def runTask(user: String)(implicit session: Session): Unit = { - if(routing.filter.filter("/" + repoName, Some(user), loadSystemSettings(), false)){ + override protected def runTask(user: String): Unit = { + val execute = Database() withSession { implicit session => + routing.filter.filter("/" + repoName, Some(user), loadSystemSettings(), false) + } + + if(execute){ val path = routing.urlPattern.r.replaceFirstIn(repoName, routing.localPath) using(Git.open(new File(Directory.GitBucketHome, path))){ git => val repository = git.getRepository @@ -155,8 +165,11 @@ class PluginGitReceivePack(repoName: String, routing: GitRepositoryRouting) extends GitCommand with SystemSettingsService { - override protected def runTask(user: String)(implicit session: Session): Unit = { - if(routing.filter.filter("/" + repoName, Some(user), loadSystemSettings(), true)){ + override protected def runTask(user: String): Unit = { + val execute = Database() withSession { implicit session => + routing.filter.filter("/" + repoName, Some(user), loadSystemSettings(), true) + } + if(execute){ val path = routing.urlPattern.r.replaceFirstIn(repoName, routing.localPath) using(Git.open(new File(Directory.GitBucketHome, path))){ git => val repository = git.getRepository diff --git a/src/main/scala/gitbucket/core/ssh/NoShell.scala b/src/main/scala/gitbucket/core/ssh/NoShell.scala index 47347fc..065bee0 100644 --- a/src/main/scala/gitbucket/core/ssh/NoShell.scala +++ b/src/main/scala/gitbucket/core/ssh/NoShell.scala @@ -1,6 +1,5 @@ package gitbucket.core.ssh -import gitbucket.core.service.SystemSettingsService import gitbucket.core.service.SystemSettingsService.SshAddress import org.apache.sshd.common.Factory import org.apache.sshd.server.{Environment, ExitCallback, Command} diff --git a/src/main/scala/gitbucket/core/util/ControlUtil.scala b/src/main/scala/gitbucket/core/util/ControlUtil.scala index a323c42..74569ac 100644 --- a/src/main/scala/gitbucket/core/util/ControlUtil.scala +++ b/src/main/scala/gitbucket/core/util/ControlUtil.scala @@ -20,6 +20,20 @@ } } + def using[A <% { def close(): Unit }, B <% { def close(): Unit }, C](resource1: A, resource2: B)(f: (A, B) => C): C = + try f(resource1, resource2) finally { + if(resource1 != null){ + ignoring(classOf[Throwable]) { + resource1.close() + } + } + if(resource2 != null){ + ignoring(classOf[Throwable]) { + resource2.close() + } + } + } + def using[T](git: Git)(f: Git => T): T = try f(git) finally git.getRepository.close() diff --git a/src/main/scala/gitbucket/core/util/Directory.scala b/src/main/scala/gitbucket/core/util/Directory.scala index de97bbb..e73bca8 100644 --- a/src/main/scala/gitbucket/core/util/Directory.scala +++ b/src/main/scala/gitbucket/core/util/Directory.scala @@ -1,11 +1,9 @@ package gitbucket.core.util import java.io.File -import ControlUtil._ -import org.apache.commons.io.FileUtils /** - * Provides directories used by GitBucket. + * Provides directory locations used by GitBucket. */ object Directory { @@ -51,6 +49,12 @@ new File(s"${RepositoryHome}/${owner}/${repository}/comments") /** + * Directory for files which are attached to issue. + */ + def getLfsDir(owner: String, repository: String): File = + new File(s"${RepositoryHome}/${owner}/${repository}/lfs") + + /** * Directory for uploaded files by the specified user. */ def getUserUploadDir(userName: String): File = new File(s"${GitBucketHome}/data/${userName}/files") @@ -73,12 +77,6 @@ def getPluginCacheDir(): File = new File(s"${TemporaryHome}/_plugins") /** - * Temporary directory which is used to create an archive to download repository contents. - */ - def getDownloadWorkDir(owner: String, repository: String, sessionId: String): File = - new File(getTemporaryDir(owner, repository), s"download/${sessionId}") - - /** * Substance directory of the wiki repository. */ def getWikiRepositoryDir(owner: String, repository: String): File = diff --git a/src/main/scala/gitbucket/core/util/FileUtil.scala b/src/main/scala/gitbucket/core/util/FileUtil.scala index f753a60..4836c23 100644 --- a/src/main/scala/gitbucket/core/util/FileUtil.scala +++ b/src/main/scala/gitbucket/core/util/FileUtil.scala @@ -62,4 +62,8 @@ "image/jpeg", "image/png", "text/plain") + + def getLfsFilePath(owner: String, repository: String, oid: String): String = + Directory.getLfsDir(owner, repository) + "/" + oid + } diff --git a/src/main/scala/gitbucket/core/util/Implicits.scala b/src/main/scala/gitbucket/core/util/Implicits.scala index d93ebe2..77767a3 100644 --- a/src/main/scala/gitbucket/core/util/Implicits.scala +++ b/src/main/scala/gitbucket/core/util/Implicits.scala @@ -24,7 +24,7 @@ implicit def context2ApiJsonFormatContext(implicit context: Context): JsonFormat.Context = JsonFormat.Context(context.baseUrl) - implicit class RichSeq[A](seq: Seq[A]) { + implicit class RichSeq[A](private val seq: Seq[A]) extends AnyVal { def splitWith(condition: (A, A) => Boolean): Seq[Seq[A]] = split(seq)(condition) @@ -40,7 +40,7 @@ } } - implicit class RichString(value: String){ + implicit class RichString(private val value: String) extends AnyVal { def replaceBy(regex: Regex)(replace: Regex.MatchData => Option[String]): String = { val sb = new StringBuilder() var i = 0 @@ -63,7 +63,7 @@ } } - implicit class RichRequest(request: HttpServletRequest){ + implicit class RichRequest(private val request: HttpServletRequest) extends AnyVal { def paths: Array[String] = (request.getRequestURI.substring(request.getContextPath.length + 1) match{ case path if path.startsWith("api/v3/repos/") => path.substring(13/* "/api/v3/repos".length */) @@ -84,7 +84,7 @@ } } - implicit class RichSession(session: HttpSession){ + implicit class RichSession(private val session: HttpSession) extends AnyVal { def getAndRemove[T](key: String): Option[T] = { val value = session.getAttribute(key).asInstanceOf[T] if(value == null){ diff --git a/src/main/scala/gitbucket/core/util/JDBCUtil.scala b/src/main/scala/gitbucket/core/util/JDBCUtil.scala index edc41ec..f94b341 100644 --- a/src/main/scala/gitbucket/core/util/JDBCUtil.scala +++ b/src/main/scala/gitbucket/core/util/JDBCUtil.scala @@ -16,7 +16,7 @@ */ object JDBCUtil { - implicit class RichConnection(conn: Connection){ + implicit class RichConnection(private val conn: Connection) extends AnyVal { def update(sql: String, params: Any*): Int = { execute(sql, params: _*){ stmt => @@ -214,8 +214,6 @@ tsort(edges).toSeq } - case class TableDependency(tableName: String, children: Seq[String]) - def tsort[A](edges: Traversable[(A, A)]): Iterable[A] = { @tailrec @@ -236,4 +234,6 @@ } } + private case class TableDependency(tableName: String, children: Seq[String]) + } diff --git a/src/main/scala/gitbucket/core/util/Notifier.scala b/src/main/scala/gitbucket/core/util/Notifier.scala index 58b9d32..86a2c88 100644 --- a/src/main/scala/gitbucket/core/util/Notifier.scala +++ b/src/main/scala/gitbucket/core/util/Notifier.scala @@ -1,6 +1,6 @@ package gitbucket.core.util -import gitbucket.core.model.{Session, Issue} +import gitbucket.core.model.{Session, Issue, Account} import gitbucket.core.model.Profile.profile._ import gitbucket.core.service.{RepositoryService, AccountService, IssuesService, SystemSettingsService} import gitbucket.core.servlet.Database @@ -11,16 +11,16 @@ import ExecutionContext.Implicits.global import org.apache.commons.mail.{DefaultAuthenticator, HtmlEmail} import org.slf4j.LoggerFactory - import gitbucket.core.controller.Context import SystemSettingsService.Smtp import ControlUtil.defining trait Notifier extends RepositoryService with AccountService with IssuesService { + def toNotify(r: RepositoryService.RepositoryInfo, issue: Issue, content: String) (msg: String => String)(implicit context: Context): Unit - protected def recipients(issue: Issue)(notify: String => Unit)(implicit session: Session, context: Context) = + protected def recipients(issue: Issue, loginAccount: Account)(notify: String => Unit)(implicit session: Session) = ( // individual repository's owner issue.userName :: @@ -33,9 +33,13 @@ getComments(issue.userName, issue.repositoryName, issue.issueId).map(_.commentedUserName) ) .distinct - .withFilter ( _ != context.loginAccount.get.userName ) // the operation in person is excluded - .foreach ( getAccountByUserName(_) filterNot (_.isGroupAccount) filterNot (LDAPUtil.isDummyMailAddress(_)) foreach (x => notify(x.mailAddress)) ) - + .withFilter ( _ != loginAccount.userName ) // the operation in person is excluded + .foreach ( + getAccountByUserName(_) + .filterNot (_.isGroupAccount) + .filterNot (LDAPUtil.isDummyMailAddress(_)) + .foreach (x => notify(x.mailAddress)) + ) } object Notifier { @@ -72,35 +76,36 @@ private val logger = LoggerFactory.getLogger(classOf[Mailer]) def toNotify(r: RepositoryService.RepositoryInfo, issue: Issue, content: String) - (msg: String => String)(implicit context: Context) = { - val database = Database() + (msg: String => String)(implicit context: Context): Unit = { + context.loginAccount.foreach { loginAccount => + val database = Database() - val f = Future { - database withSession { implicit session => - defining( - s"[${r.name}] ${issue.title} (#${issue.issueId})" -> - msg(Markdown.toHtml( - markdown = content, - repository = r, - enableWikiLink = false, - enableRefsLink = true, - enableAnchor = false, - enableLineBreaks = false - ))) { case (subject, msg) => - recipients(issue) { to => - send(to, subject, msg) - } + val f = Future { + database withSession { implicit session => + defining( + s"[${r.name}] ${issue.title} (#${issue.issueId})" -> + msg(Markdown.toHtml( + markdown = content, + repository = r, + enableWikiLink = false, + enableRefsLink = true, + enableAnchor = false, + enableLineBreaks = false + )) + ) { case (subject, msg) => + recipients(issue, loginAccount) { to => send(to, subject, msg, loginAccount) } + } } + "Notifications Successful." } - "Notifications Successful." - } - f.onComplete { - case Success(s) => logger.debug(s) - case Failure(t) => logger.error("Notifications Failed.", t) + f.onComplete { + case Success(s) => logger.debug(s) + case Failure(t) => logger.error("Notifications Failed.", t) + } } } - def send(to: String, subject: String, msg: String)(implicit context: Context): Unit = { + def send(to: String, subject: String, msg: String, loginAccount: Account): Unit = { val email = new HtmlEmail email.setHostName(smtp.host) email.setSmtpPort(smtp.port.get) @@ -113,9 +118,13 @@ email.setSslSmtpPort(smtp.port.get.toString) } } + smtp.starttls.foreach { starttls => + email.setStartTLSEnabled(starttls) + email.setStartTLSRequired(starttls) + } smtp.fromAddress - .map (_ -> smtp.fromName.getOrElse(context.loginAccount.get.userName)) - .orElse (Some("notifications@gitbucket.com" -> context.loginAccount.get.userName)) + .map (_ -> smtp.fromName.getOrElse(loginAccount.userName)) + .orElse (Some("notifications@gitbucket.com" -> loginAccount.userName)) .foreach { case (address, name) => email.setFrom(address, name) } diff --git a/src/main/scala/gitbucket/core/util/StringUtil.scala b/src/main/scala/gitbucket/core/util/StringUtil.scala index 768c407..da7dc50 100644 --- a/src/main/scala/gitbucket/core/util/StringUtil.scala +++ b/src/main/scala/gitbucket/core/util/StringUtil.scala @@ -1,14 +1,23 @@ package gitbucket.core.util import java.net.{URLDecoder, URLEncoder} + import org.mozilla.universalchardet.UniversalDetector import ControlUtil._ import org.apache.commons.io.input.BOMInputStream import org.apache.commons.io.IOUtils +import org.apache.commons.codec.binary.Base64 + import scala.util.control.Exception._ object StringUtil { + private lazy val BlowfishKey = { + // last 4 numbers in current timestamp + val time = System.currentTimeMillis.toString + time.substring(time.length - 4) + } + def sha1(value: String): String = defining(java.security.MessageDigest.getInstance("SHA-1")){ md => md.update(value.getBytes) @@ -21,6 +30,20 @@ md.digest.map(b => "%02x".format(b)).mkString } + def encodeBlowfish(value: String): String = { + val spec = new javax.crypto.spec.SecretKeySpec(BlowfishKey.getBytes(), "Blowfish") + val cipher = javax.crypto.Cipher.getInstance("Blowfish") + cipher.init(javax.crypto.Cipher.ENCRYPT_MODE, spec) + new String(Base64.encodeBase64(cipher.doFinal(value.getBytes("UTF-8"))), "UTF-8") + } + + def decodeBlowfish(value: String): String = { + val spec = new javax.crypto.spec.SecretKeySpec(BlowfishKey.getBytes(), "Blowfish") + val cipher = javax.crypto.Cipher.getInstance("Blowfish") + cipher.init(javax.crypto.Cipher.DECRYPT_MODE, spec) + new String(cipher.doFinal(Base64.decodeBase64(value)), "UTF-8") + } + def urlEncode(value: String): String = URLEncoder.encode(value, "UTF-8").replace("+", "%20") def urlDecode(value: String): String = URLDecoder.decode(value, "UTF-8") diff --git a/src/main/scala/gitbucket/core/view/helpers.scala b/src/main/scala/gitbucket/core/view/helpers.scala index d29248a..beff9d5 100644 --- a/src/main/scala/gitbucket/core/view/helpers.scala +++ b/src/main/scala/gitbucket/core/view/helpers.scala @@ -161,7 +161,7 @@ } import scala.util.matching.Regex._ - implicit class RegexReplaceString(s: String) { + implicit class RegexReplaceString(private val s: String) extends AnyVal { def replaceAll(pattern: String, replacer: (Match) => String): String = { pattern.r.replaceAllIn(s, (m: Match) => replacer(m).replace("$", "\\$")) } @@ -297,7 +297,7 @@ /** * Implicit conversion to add mkHtml() to Seq[Html]. */ - implicit class RichHtmlSeq(seq: Seq[Html]) { + implicit class RichHtmlSeq(private val seq: Seq[Html]) extends AnyVal { def mkHtml(separator: String) = Html(seq.mkString(separator)) def mkHtml(separator: scala.xml.Elem) = Html(seq.mkString(separator.toString)) } diff --git a/src/main/twirl/gitbucket/core/account/application.scala.html b/src/main/twirl/gitbucket/core/account/application.scala.html index fd3b8e7..a065e66 100644 --- a/src/main/twirl/gitbucket/core/account/application.scala.html +++ b/src/main/twirl/gitbucket/core/account/application.scala.html @@ -10,7 +10,7 @@ @if(personalTokens.isEmpty && gneratedToken.isEmpty){ No tokens. } else { - Tokens you have generated that can be used to access the GitBucket API. + Tokens you have generated which can be used to access the GitBucket API.
} @gneratedToken.map { case (token, tokenString) => @@ -42,7 +42,7 @@
-

What's this token for?

+

What is this token for?

diff --git a/src/main/twirl/gitbucket/core/account/edit.scala.html b/src/main/twirl/gitbucket/core/account/edit.scala.html index 21377e5..757e068 100644 --- a/src/main/twirl/gitbucket/core/account/edit.scala.html +++ b/src/main/twirl/gitbucket/core/account/edit.scala.html @@ -37,6 +37,11 @@ +
+ + + +
diff --git a/src/main/twirl/gitbucket/core/account/group.scala.html b/src/main/twirl/gitbucket/core/account/group.scala.html index 6be84c9..725581f 100644 --- a/src/main/twirl/gitbucket/core/account/group.scala.html +++ b/src/main/twirl/gitbucket/core/account/group.scala.html @@ -22,6 +22,12 @@
+ +
+ +
+
+
@gitbucket.core.helper.html.uploadavatar(account)
diff --git a/src/main/twirl/gitbucket/core/account/main.scala.html b/src/main/twirl/gitbucket/core/account/main.scala.html index 8d68399..4cdd5e5 100644 --- a/src/main/twirl/gitbucket/core/account/main.scala.html +++ b/src/main/twirl/gitbucket/core/account/main.scala.html @@ -12,6 +12,9 @@
+ @account.description.map{ description => +

@description

+ } @if(account.url.isDefined){

@account.url diff --git a/src/main/twirl/gitbucket/core/account/newrepo.scala.html b/src/main/twirl/gitbucket/core/account/newrepo.scala.html index 2cf2bc4..6f6397b 100644 --- a/src/main/twirl/gitbucket/core/account/newrepo.scala.html +++ b/src/main/twirl/gitbucket/core/account/newrepo.scala.html @@ -6,7 +6,7 @@

Create a new repository

- A repository contains all the files for your project, including the revision history. + A repository contains all the files for your project including the revision history.

diff --git a/src/main/twirl/gitbucket/core/account/register.scala.html b/src/main/twirl/gitbucket/core/account/register.scala.html index e54b5d8..169102e 100644 --- a/src/main/twirl/gitbucket/core/account/register.scala.html +++ b/src/main/twirl/gitbucket/core/account/register.scala.html @@ -33,6 +33,11 @@
+
+ + + +
diff --git a/src/main/twirl/gitbucket/core/admin/system.scala.html b/src/main/twirl/gitbucket/core/admin/system.scala.html index b2a582d..c78ad6d 100644 --- a/src/main/twirl/gitbucket/core/admin/system.scala.html +++ b/src/main/twirl/gitbucket/core/admin/system.scala.html @@ -36,8 +36,8 @@

The base URL is used for redirect, notification email, git repository URL box and more. - If the base URL is empty, GitBucket generates URL from request information. - You can use this property to adjust URL difference between the reverse proxy and GitBucket. + If the base URL is empty, GitBucket generates URL from the request information. + You can use this property to adjust to URL differences between the reverse proxy and GitBucket.

@@ -59,19 +59,19 @@
- +
@@ -82,7 +82,7 @@
- +
- +
@@ -178,7 +176,7 @@
- +
@@ -259,49 +257,56 @@
- +
- +
- +
- +
- +
- + +
+ +
+
+
+
- +
@@ -311,11 +316,23 @@
- -

- Enable notification not only SMTP configuration if you want to send notification email. -

+ + + + @* +
+ +
+ +
+ + +
+
+ *@
@@ -332,6 +349,7 @@ var user = $('#smtpUser' ).val(); var password = $('#smtpPassword').val(); var ssl = $('#smtpSsl' ).prop('checked'); + var starttls = $('#smtpStarttls').prop('checked'); var fromAddress = $('#fromAddress' ).val(); var fromName = $('#fromName' ).val(); var testAddress = $('#testAddress' ).val(); @@ -349,6 +367,7 @@ 'smtp.user': user, 'smtp.password': password, 'smtp.ssl': ssl, + 'smtp.starttls': starttls, 'smtp.fromAddress': fromAddress, 'smtp.fromName': fromName, 'testAddress': testAddress diff --git a/src/main/twirl/gitbucket/core/admin/user.scala.html b/src/main/twirl/gitbucket/core/admin/user.scala.html index 843354f..279f953 100644 --- a/src/main/twirl/gitbucket/core/admin/user.scala.html +++ b/src/main/twirl/gitbucket/core/admin/user.scala.html @@ -66,6 +66,13 @@
+
+ +
+ +
+ +
diff --git a/src/main/twirl/gitbucket/core/admin/usergroup.scala.html b/src/main/twirl/gitbucket/core/admin/usergroup.scala.html index a656342..e8b03cf 100644 --- a/src/main/twirl/gitbucket/core/admin/usergroup.scala.html +++ b/src/main/twirl/gitbucket/core/admin/usergroup.scala.html @@ -25,6 +25,10 @@
+ + +
+
@gitbucket.core.helper.html.uploadavatar(account)
diff --git a/src/main/twirl/gitbucket/core/issues/create.scala.html b/src/main/twirl/gitbucket/core/issues/create.scala.html index a418a2b..f72df6b 100644 --- a/src/main/twirl/gitbucket/core/issues/create.scala.html +++ b/src/main/twirl/gitbucket/core/issues/create.scala.html @@ -2,6 +2,7 @@ milestones: List[gitbucket.core.model.Milestone], labels: List[gitbucket.core.model.Label], isManageable: Boolean, + content: String, repository: gitbucket.core.service.RepositoryService.RepositoryInfo)(implicit context: gitbucket.core.controller.Context) @import gitbucket.core.view.helpers @gitbucket.core.html.main(s"New Issue - ${repository.owner}/${repository.name}", Some(repository)){ @@ -13,7 +14,7 @@ @gitbucket.core.helper.html.preview( repository = repository, - content = "", + content = content, enableWikiLink = false, enableRefsLink = true, enableLineBreaks = true, diff --git a/src/main/twirl/gitbucket/core/issues/labels/edit.scala.html b/src/main/twirl/gitbucket/core/issues/labels/edit.scala.html index 34b2888..b87957c 100644 --- a/src/main/twirl/gitbucket/core/issues/labels/edit.scala.html +++ b/src/main/twirl/gitbucket/core/issues/labels/edit.scala.html @@ -6,7 +6,7 @@
- +
- - + + diff --git a/src/main/twirl/gitbucket/core/pulls/compare.scala.html b/src/main/twirl/gitbucket/core/pulls/compare.scala.html index 06d00ff..d97b36f 100644 --- a/src/main/twirl/gitbucket/core/pulls/compare.scala.html +++ b/src/main/twirl/gitbucket/core/pulls/compare.scala.html @@ -7,6 +7,7 @@ forkedId: String, sourceId: String, commitId: String, + content: String, repository: gitbucket.core.service.RepositoryService.RepositoryInfo, originRepository: gitbucket.core.service.RepositoryService.RepositoryInfo, forkedRepository: gitbucket.core.service.RepositoryService.RepositoryInfo, @@ -59,7 +60,7 @@ @gitbucket.core.helper.html.preview( repository = repository, - content = "", + content = content, enableWikiLink = false, enableRefsLink = true, enableLineBreaks = true, diff --git a/src/main/twirl/gitbucket/core/pulls/conversation.scala.html b/src/main/twirl/gitbucket/core/pulls/conversation.scala.html index 94dcbbb..85e862c 100644 --- a/src/main/twirl/gitbucket/core/pulls/conversation.scala.html +++ b/src/main/twirl/gitbucket/core/pulls/conversation.scala.html @@ -34,7 +34,7 @@
Pull request successfully merged and closed
- You're all set-the @pullreq.requestBranch branch can be safely deleted. + You're all set. The @pullreq.requestBranch branch can now be safely deleted.
} diff --git a/src/main/twirl/gitbucket/core/repo/blob.scala.html b/src/main/twirl/gitbucket/core/repo/blob.scala.html index 9ae999d..9d5406a 100644 --- a/src/main/twirl/gitbucket/core/repo/blob.scala.html +++ b/src/main/twirl/gitbucket/core/repo/blob.scala.html @@ -4,7 +4,8 @@ content: gitbucket.core.util.JGitUtil.ContentInfo, latestCommit: gitbucket.core.util.JGitUtil.CommitInfo, hasWritePermission: Boolean, - isBlame: Boolean)(implicit context: gitbucket.core.controller.Context) + isBlame: Boolean, + isLfsFile: Boolean)(implicit context: gitbucket.core.controller.Context) @import gitbucket.core.view.helpers @gitbucket.core.html.main(s"${(repository.name :: pathList).mkString("/")} at ${helpers.encodeRefName(branch)} - ${repository.owner}/${repository.name}", Some(repository)) { @gitbucket.core.html.menu("files", repository){ @@ -45,6 +46,9 @@ @section / } } + @if(isLfsFile){ + LFS + }
@helpers.avatar(latestCommit, 28) @@ -92,12 +96,13 @@ } } } -