Update changelog, skip docker for now until it is ready

CHANGELOG.md

@@ -4,7 +4,7 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
-## [v0.6.0] - 2023-03-09
+## [v0.6.0] - 2023-03-15
 
 - Configurable destinations for downloads
   - Multiple destination directories can be configured
@@ -12,10 +12,11 @@ All notable changes to this project will be documented in this file.
 - When downloading from a playlist, show the total number of videos and how many have been downloaded
 - Show version in web UI
 - Improve index page (show URL of queued downloads instead of nothing)
-- Add docker support
 - Fixes and improvements to capturing output info and showing it in the UI
+- Show all log output in the popup
 - Fixes to handling of queued downloads
 - Fix portable mode to look in binary directory, not current directory
+- Automatically cleanup download list, removing old entries automatically
 
 ## [v0.5.5] - 2022-04-09
 

README.md

@@ -16,9 +16,10 @@ A frontend to youtube-dl (or compatible forks, like yt-dlp) to download videos w
 
 ## Binaries
 
-Binaries are available at https://github.com/tardisx/gropple/releases
+Binaries are available at <https://github.com/tardisx/gropple/releases>
 
-Gropple will automatically check for available updates and prompt you to upgrade.
+Gropple will automatically check for available updates and prompt you to
+upgrade.
 
 ## Running
 
@@ -32,66 +33,75 @@ interface. The address will be printed after startup:
 
 ## Using
 
-Bring up `http://localhost:6283` (or your configured address) in your browser. You 
-should see a link to the bookmarklet at the top of the screen, and the list of
-downloads (currently empty).
+Bring up `http://localhost:6283` (or your configured address) in your browser.
+You should see a link to the bookmarklet at the top of the screen, and the list
+of downloads (currently empty).
 
-Drag the bookmarklet to your favourites bar, or otherwise bookmark it as you 
-see fit. Any kind of browser bookmark should work. The bookmarklet contains
-embedded javascript to pass the URL of whatever page you are currently on back
-to gropple.
+Drag the bookmarklet to your favourites bar, or otherwise bookmark it as you see
+fit. Any kind of browser bookmark should work. The bookmarklet contains embedded
+javascript to pass the URL of whatever page you are currently on back to
+gropple.
 
-So, whenever you are on a page with a video you would like to download just 
-click the bookmarklet.
+Whenever you are on a page with a video you would like to download just click
+the bookmarklet.
 
-A popup window will appear. Choose a download profile and the download will start.
-The status will be shown in the window, updating in real time.
+A popup window will appear. Choose a download profile and the download will
+start. The status will be shown in the window, updating in real time.
 
-You may close this window at any time without stopping the download, the status 
+You may close this window at any time without stopping the download, the status
 of all downloads is available on the index page.
 
 ## Configuration
 
-Click the "config" link on the index page to configure gropple. The default options 
-are fine if you are running on your local machine. If you are running it remotely
-you will need to set the "server address" to ensure the bookmarklet has the correct
-URL in it.
+Click the "config" link on the index page to configure gropple. The default
+options are fine if you are running on your local machine. If you are running it
+remotely you will need to set the "server address" to ensure the bookmarklet has
+the correct URL in it.
 
 ### Configuring Downloaders
 
-Gropple's default configuration uses the original youtube-dl and has two profiles set
-up, one for downloading video, the other for downloading audio (mp3).
+Gropple's default configuration uses `yt-dlp` and has two profiles set up, one
+for downloading video, the other for downloading audio (mp3).
 
-Note that gropple does not include any downloaders, you have to install them separately.
+Note that gropple does not include any downloaders, you have to install them
+separately.
 
-If you would like to use a youtube-dl fork (like [yt-dlp](https://github.com/yt-dlp/yt-dlp))
-or change the options, you can do so on the right hand side. Create as many profiles as you 
-wish, whenever you start a download you can choose the appropriate profile.
+If you would like to use a youtube-dl compatible fork or change the options you
+can do so on the right hand side. Create as many profiles as you wish, whenever
+you start a download you can choose the appropriate profile.
 
-Note that the command arguments must each be specified separately - see the default configuration
-for an example.
+Note that the command arguments must each be specified separately - see the
+default configuration for an example.
 
-While gropple will use your `PATH` to find the executable, you can also specify a full path
-instead. Note that any tools that the downloader calls itself (for instance, ffmpeg) will 
-probably need to be available on your path.
+While gropple will use your `PATH` to find the executable, you can also specify
+a full path instead. Note that any tools that the downloader calls itself (for
+instance, `ffmpeg`) will need to be available on your path.
+
+### Alternate destinations
+
+Gropple supports adding additional optional destinations. By default, all
+downloads will be stored in the main download path specified in the config. You
+can also add one or more destinations, and you can choose one of these
+destinations when queueing a new download, or while it is still downloading from
+the popup.
+
+The file will be moved after downloading is complete.
 
 ## Portable mode
 
-If you'd like to use gropple from a USB stick or similar, copy the config file from
-it's default location (shown when you start gropple) to the same location as the binary, and rename it to `gropple.yml`. 
-
-If that file is present in the same directory as the binary, it will be used instead.
+If you'd like to use gropple from a USB stick or similar, copy the config file
+from its default location (shown when you start gropple) to the same location as
+the binary, and rename it to `gropple.yml`.
 
 ## Problems
 
-Most download problems are probably diagnosable via the log - check in the popup window and scroll
-the log down to the bottom. The most common problem is that youtube-dl cannot be found, or its
-dependency (like ffmpeg) cannot be found on your path. 
+Many download problems are diagnosable via the log - check in the popup window
+and scroll the log down to the bottom. The most common problem is that `yt-dlp`
+cannot be found, or its dependency (like `ffmpeg`) cannot be found on your path.
 
 For other problems, please file an issue on github.
 
 ## TODO
 
-Many things. Please raise an issue after checking the [currently open issues](https://github.com/tardisx/gropple/issues).
-
-
+Many things. Please raise an issue after checking the [currently open
+issues](https://github.com/tardisx/gropple/issues).

build_release.pl

@@ -40,6 +40,7 @@ foreach my $type (keys %build) {
 }
 
 # now docker
+exit 0;
 $ENV{VERSION}="$version";
 system "docker-compose", "-f", "docker-compose.build.yml", "build";
 system "docker", "tag", "tardisx/gropple:$version", "tardisx/gropple:latest";

config/config.go

@@ -94,7 +94,6 @@ func (cs *ConfigService) LoadDefaultConfig() {
 
 	cs.Config = &defaultConfig
 
-	return
 }
 
 func (c *Config) ProfileCalled(name string) *DownloadProfile {
@@ -319,6 +318,9 @@ func (cs *ConfigService) WriteConfig() {
 	}
 	defer file.Close()
 
-	file.Write(s)
+	_, err = file.Write(s)
+	if err != nil {
+		log.Fatalf("could not write config file %s: %s", path, err)
+	}
 	file.Close()
 }

download/download.go

@@ -83,7 +83,7 @@ func (m *Manager) ManageQueue() {
 
 		m.startQueued(m.MaxPerDomain)
 		m.moveToDest()
-		// m.cleanup()
+		m.cleanup()
 		m.Lock.Unlock()
 
 		time.Sleep(time.Second)
@@ -167,16 +167,17 @@ func (m *Manager) startQueued(maxRunning int) {
 }
 
 // cleanup removes old downloads from the list. Hardcoded to remove them one hour
-// completion.
-func (m *Manager) XXXcleanup() {
+// completion. Expects the Manager to be locked.
+func (m *Manager) cleanup() {
 	newDLs := []*Download{}
 	for _, dl := range m.Downloads {
-
+		dl.Lock.Lock()
 		if dl.Finished && time.Since(dl.FinishedTS) > time.Duration(time.Hour) {
 			// do nothing
 		} else {
 			newDLs = append(newDLs, dl)
 		}
+		dl.Lock.Unlock()
 
 	}
 	m.Downloads = newDLs
@@ -250,7 +251,10 @@ func (dl *Download) Stop() {
 	dl.Lock.Lock()
 	defer dl.Lock.Unlock()
 	dl.Log = append(dl.Log, "aborted by user")
-	dl.Process.Kill()
+	err := dl.Process.Kill()
+	if err != nil {
+		log.Printf("could not send kill to process: %s", err)
+	}
 }
 
 // domain returns a domain for this Download. Download should be locked.
@@ -335,19 +339,30 @@ func (dl *Download) Begin() {
 	}()
 
 	wg.Wait()
-	cmd.Wait()
 
+	err = cmd.Wait()
 	dl.Lock.Lock()
 
-	log.Printf("Process finished for id: %d (%v)", dl.Id, cmd)
+	if err != nil {
+		log.Printf("process failed for id: %d: %s", dl.Id, err)
 
-	dl.State = STATE_COMPLETE
-	dl.Finished = true
-	dl.FinishedTS = time.Now()
-	dl.ExitCode = cmd.ProcessState.ExitCode()
-
-	if dl.ExitCode != 0 {
 		dl.State = STATE_FAILED
+		dl.Finished = true
+		dl.FinishedTS = time.Now()
+		dl.ExitCode = cmd.ProcessState.ExitCode()
+
+	} else {
+
+		log.Printf("process finished for id: %d (%v)", dl.Id, cmd)
+
+		dl.State = STATE_COMPLETE
+		dl.Finished = true
+		dl.FinishedTS = time.Now()
+		dl.ExitCode = cmd.ProcessState.ExitCode()
+
+		if dl.ExitCode != 0 {
+			dl.State = STATE_FAILED
+		}
 	}
 	dl.Lock.Unlock()
 

main.go

@@ -24,7 +24,7 @@ var dm *download.Manager
 var configService *config.ConfigService
 
 var versionInfo = version.Manager{
-	VersionInfo: version.Info{CurrentVersion: "v0.6.0-alpha.3"},
+	VersionInfo: version.Info{CurrentVersion: "v0.6.0-alpha.4"},
 }
 
 //go:embed web
@@ -101,7 +101,10 @@ func main() {
 	// check for a new version every 4 hours
 	go func() {
 		for {
-			versionInfo.UpdateGitHubVersion()
+			err := versionInfo.UpdateGitHubVersion()
+			if err != nil {
+				log.Printf("could not get version info: %s", err)
+			}
 			time.Sleep(time.Hour * 4)
 		}
 	}()
@@ -120,7 +123,10 @@ func main() {
 func versionRESTHandler(w http.ResponseWriter, r *http.Request) {
 	if versionInfo.GetInfo().GithubVersionFetched {
 		b, _ := json.Marshal(versionInfo.GetInfo())
-		w.Write(b)
+		_, err := w.Write(b)
+		if err != nil {
+			log.Printf("could not write to client: %s", err)
+		}
 	} else {
 		w.WriteHeader(400)
 	}
@@ -172,7 +178,10 @@ func staticHandler(w http.ResponseWriter, r *http.Request) {
 			return
 		}
 		w.WriteHeader(http.StatusOK)
-		io.Copy(w, f)
+		_, err = io.Copy(w, f)
+		if err != nil {
+			log.Printf("could not write to client: %s", err)
+		}
 		return
 	}
 	w.WriteHeader(http.StatusNotFound)
@@ -208,13 +217,19 @@ func configRESTHandler(w http.ResponseWriter, r *http.Request) {
 			errorRes := errorResponse{Success: false, Error: err.Error()}
 			errorResB, _ := json.Marshal(errorRes)
 			w.WriteHeader(400)
-			w.Write(errorResB)
+			_, err = w.Write(errorResB)
+			if err != nil {
+				log.Printf("could not write to client: %s", err)
+			}
 			return
 		}
 		configService.WriteConfig()
 	}
 	b, _ := json.Marshal(configService.Config)
-	w.Write(b)
+	_, err := w.Write(b)
+	if err != nil {
+		log.Printf("could not write config to client: %s", err)
+	}
 }
 
 func fetchInfoOneRESTHandler(w http.ResponseWriter, r *http.Request) {
@@ -256,7 +271,10 @@ func fetchInfoOneRESTHandler(w http.ResponseWriter, r *http.Request) {
 				errorRes := errorResponse{Success: false, Error: err.Error()}
 				errorResB, _ := json.Marshal(errorRes)
 				w.WriteHeader(400)
-				w.Write(errorResB)
+				_, err = w.Write(errorResB)
+				if err != nil {
+					log.Printf("could not write to client: %s", err)
+				}
 				return
 			}
 
@@ -275,7 +293,10 @@ func fetchInfoOneRESTHandler(w http.ResponseWriter, r *http.Request) {
 
 				succRes := successResponse{Success: true, Message: "download started"}
 				succResB, _ := json.Marshal(succRes)
-				w.Write(succResB)
+				_, err = w.Write(succResB)
+				if err != nil {
+					log.Printf("could not write to client: %s", err)
+				}
 				return
 			}
 
@@ -290,7 +311,10 @@ func fetchInfoOneRESTHandler(w http.ResponseWriter, r *http.Request) {
 
 				succRes := successResponse{Success: true, Message: "destination changed"}
 				succResB, _ := json.Marshal(succRes)
-				w.Write(succResB)
+				_, err = w.Write(succResB)
+				if err != nil {
+					log.Printf("could not write to client: %s", err)
+				}
 				return
 			}
 
@@ -299,7 +323,10 @@ func fetchInfoOneRESTHandler(w http.ResponseWriter, r *http.Request) {
 				thisDownload.Stop()
 				succRes := successResponse{Success: true, Message: "download stopped"}
 				succResB, _ := json.Marshal(succRes)
-				w.Write(succResB)
+				_, err = w.Write(succResB)
+				if err != nil {
+					log.Printf("could not write to client: %s", err)
+				}
 				return
 			}
 		}
@@ -310,7 +337,10 @@ func fetchInfoOneRESTHandler(w http.ResponseWriter, r *http.Request) {
 
 		b, _ := json.Marshal(thisDownload)
 
-		w.Write(b)
+		_, err = w.Write(b)
+		if err != nil {
+			log.Printf("could not write to client: %s", err)
+		}
 		return
 	} else {
 		http.NotFound(w, r)
@@ -323,7 +353,10 @@ func fetchInfoRESTHandler(w http.ResponseWriter, r *http.Request) {
 	if err != nil {
 		panic(err)
 	}
-	w.Write(b)
+	_, err = w.Write(b)
+	if err != nil {
+		log.Printf("could not write to client: %s", err)
+	}
 }
 
 func fetchHandler(w http.ResponseWriter, r *http.Request) {
@@ -377,18 +410,14 @@ func fetchHandler(w http.ResponseWriter, r *http.Request) {
 		}
 
 		// create the new download
-		log.Print("creating")
 		newDL := download.NewDownload(url[0], configService.Config)
-		log.Print("adding")
 		dm.AddDownload(newDL)
-		log.Print("done")
 
 		t, err := template.ParseFS(webFS, "web/layout.tmpl", "web/popup.html")
 		if err != nil {
 			panic(err)
 		}
 
-		log.Print("lock dl")
 		newDL.Lock.Lock()
 		defer newDL.Lock.Unlock()
 
@@ -398,6 +427,5 @@ func fetchHandler(w http.ResponseWriter, r *http.Request) {
 		if err != nil {
 			panic(err)
 		}
-		log.Print("unlock dl because rendered")
 	}
 }

version/version.go

@@ -26,7 +26,6 @@ type Manager struct {
 }
 
 func (m *Manager) GetInfo() Info {
-	// log.Print("getting info... b4 lock")
 	m.lock.Lock()
 	defer m.lock.Unlock()
 

web/popup.html

@@ -34,11 +34,11 @@
         </table>
         <p>You can close this window and your download will continue. Check the <a href="/" target="_gropple_status">Status page</a> to see all downloads in progress.</p>
         {{ if .canStop }}
-        <button x-show="state=='downloading'" class="pure-button" @click="stop()">stop</button>
+        <button x-show="state=='Downloading'" class="pure-button" @click="stop()">stop</button>
         {{ end }}
         <div>
             <h4>Logs</h4>
-            <pre x-text="log">
+            <pre x-text="log" style="height: auto;">
             </pre>
         </div>
     </div>