README.md

@@ -1,41 +1,67 @@
 # noneroll
 
-This Google Apps Script moves your low priority (promotional, social, political, etc.) emails in Gmail to a custom label, and sends you a synopsis of recieved emails every morning for review.
+This Google Apps Script moves your low priority (promotional, social, political, etc.) emails in Gmail to a custom label and sends you a daily synopsis of received emails for review. Basically, this is `unroll.me` without any third-party access or [creepy data brokering](https://archive.is/30hj9).
 
-Basically, this is `unroll.me` without any 3rd party access or [creepy data brokering](https://www.nytimes.com/2017/04/24/technology/personal-data-firm-slice-unroll-me-backlash-uber.html).
+## New Features & Improvements
+
+- **Global Constants:**  
+  Configure the script easily by setting global constants for your spreadsheet ID, sheet name, Gmail label, and email digest time range.
+  
+- **Performance Enhancements:**  
+  The script now uses JavaScript Sets and batch updates (instead of multiple `appendRow` calls) to handle larger datasets more efficiently. My personal instance of this script filters messages from almost 1300 email addresses.
+  
+- **Automatic Deduplication:**  
+  A new `dedupeEmails` function is included to automatically remove duplicate email addresses from your spreadsheet. You can schedule this to run periodically.
+    
+- **Improved Unsubscribe Link Extraction & HTML Template:**  
+  The unsubscribe link extraction has been refined, and the HTML template now uses centralized CSS.
+  
+- **Modern Code Practices:**  
+  The script has been updated with ES6 syntax (`const`, `let`, for‑of loops) for improved clarity and maintainability.
 
 ## Configuration
 
-1. Create a new spreadsheet in Google Sheets. Name it whatever you'd like.
+1. **Create a Google Sheet:**  
+   Create a new spreadsheet in Google Sheets. This spreadsheet will be used to keep track of low priority email addresses.
 
-  - This spreadsheet will be used to keep track of low priority email addresses.
+2. **Create a Gmail Label:**  
+   Create a new Gmail label (e.g., `zbulk`).  
+   If you use a different label, update the `ZBULK_LABEL` constant in the script accordingly.
 
-2. Create a new Gmail label. Mine is called `zbulk`, but you are more than welcome to be creative.
+3. **Set Up the Google Apps Script Project:**
+   - Create a new Google Apps Script project.
+   - Copy the updated code into the project's `Code.gs` file.
+   - Replace the placeholder `SHEET_ID` (currently set to `'sheetID'`) with the actual ID of the spreadsheet (the string between `spreadsheets/d/` and `/edit` in your sheet's URL).
+   - Update `SHEET_NAME` if your target sheet name differs from the default ("Sheet1").
 
-  - Should you decide on a different label, replace every instance of `zbulk` in the `noneroll.gs` file to whatever name you have choosen.
+4. **Configure the HTML Template:**
+   - Include the updated HTML file named `email.html` in your project. This file controls the appearance of your daily digest.
 
-3. Create a new Google Apps Script project.
+5. **Set Up Triggers:**  
+   Configure time-driven triggers for:
+   - **`arch`:** Every 15 minutes (or as desired)  
+     Archives, labels, and marks low priority emails as read.
+   - **`addEmail`:** Daily (e.g., between midnight and 1am)  
+     Batches and appends new email addresses to your spreadsheet.
+   - **`noneroll`:** Daily (e.g., between 5 and 6am)  
+     Sends a digest email summarizing emails from the past 24 hours.
+   - **`dedupeEmails`:** Weekly (every Sunday, between 11pm and midnight)  
+     Automatically deduplicates the email list in your spreadsheet.
 
-  - Copy code from `noneroll.gs` to the project's `Code.gs` file.
+## Usage & Maintenance
-  - Replace `sheetID` with the ID of the sheet (everything after `spreadsheets/d/` and before `/edit#` in the sheet's URL) that you created in Step 1.
 
-4. Create project triggers. I am currently using the following settings:
+- **Tagging Emails:**  
+  Tag any emails you wish to include in your low priority digest with your chosen Gmail label (e.g., `zbulk`).
 
-  - `arch` every 15 mins.
+- **Spreadsheet Data:**  
-    - Archives and labels email.
+  Column A of your spreadsheet will be populated with the email addresses of low priority senders.  
-  - `addEmail` between midnight and 1am.
+  The script’s `addEmail` function should prevent duplicate entries, but the `dedupeEmails` function will periodically clean up any accidental duplicates.
-    - Adds labeled email addresses to spreadsheet.
-  - `noneroll` between 5 and 6am.
-    - Sends digest of emails from past 24 hours.
 
-## Notes
+- **Disabling Low Priority Flag:**  
+  To stop an email from being flagged as low priority, remove its corresponding email address from your spreadsheet and (for safety) remove the label from any related messages in Gmail.
 
-- Tag any emails that you wish to add to your low priority email digest with `zbulk` (or your custom label).
+- **Performance Considerations:**  
-- Column A of your spreadhsheet will be populated with the email addresses of mail you consider low priority.
+  The updated script handles thousands of entries efficiently. For extremely large datasets, consider archiving older data or scheduling deduplication more frequently.
-- It is a good idea to occasionally open your spreadsheet and select *"Data > Remove duplicates"* from the menu to prevent email addresses from appearing multiple times in your spreadsheet.
-  - Pull requests to fix this (see: automagically deduping email addresses) are welcome.
-- If you want to stop mail from being flagged as low priority, remove the corresponding email address(es) from your spreadsheet and (to be safe) remove the label from any messages in `zbulk` (or your custom label).
-- Low priority emails will be archived, labeled, and marked as read every 15 minutes.
 
 ## Credits
 

email.html

@@ -1,40 +1,85 @@
 <!DOCTYPE html>
 <html>
 <head>
-  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+  <meta charset="utf-8">
-  <meta name="viewport" content="width=device-width; initial-scale=1.0; maximum-scale=1.0;">
+  <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0">
   <meta property="og:title" content="Bulk Emails">
   <title>Bulk Email Summary</title>
+  <style>
+    body {
+      font-family: 'Atkinson Hyperlegible', Helvetica, Arial, sans-serif;
+      color: #161616;
+      margin: 0;
+      padding: 1em;
+      background-color: #f9f9f9;
+    }
+    table {
+      width: 100%;
+      border-collapse: collapse;
+      margin-bottom: 1em;
+    }
+    td {
+      vertical-align: top;
+      padding: 1em;
+      border-bottom: 1px solid #ddd;
+    }
+    .email-from h4 {
+      margin: 0;
+      padding-top: 12px;
+      font-weight: normal;
+    }
+    .email-from a {
+      color: #3f8abf;
+      text-decoration: none;
+    }
+    .email-time {
+      margin: 0;
+      color: #555;
+    }
+    .unsubscribe a {
+      display: inline-block;
+      color: #fff;
+      padding: 0.25em 0.5em;
+      background-color: #3f8abf;
+      border-radius: 5px;
+      text-decoration: none;
+      font-size: 0.9em;
+    }
+    .email-subject a {
+      font-size: 1.5em;
+      color: #161616;
+      text-decoration: none;
+    }
+  </style>
 </head>
 
 <body>
-  <table border="0" cellspacing="1em" cellpadding="1em">
+  <table>
     <tbody>
       <? for (var i = 0; i < data.length; i++) { ?>
         <tr>
-          <td style='margin: 0 1em 0 0;'>
+          <td>
-            <h4 style="margin: 0; padding-top: 12px;font-weight:normal">
+            <div class="email-from">
-              <a style="color:rgb(63,138,191) !important;text-decoration:none;"
+              <h4>
-              href="mailto:<?= data[i].email ?>"><?= data[i].from ?></a>
+                <a href="mailto:<?= data[i].email ?>"><?= data[i].from ?></a>
               </h4>
-            <p style="margin: 0;">
+            </div>
+            <p class="email-time">
               at <?= Utilities.formatDate(data[i].date, Session.getScriptTimeZone(), "HH:mm 'on' MMM d") ?>
             </p>
-            <p style="margin: 0;">
+            <? if (data[i].uns) { ?>
-              <? if(data[i].uns) { ?>
+            <p class="unsubscribe">
-                <a style="display:inline-block;color:rgb(255,255,255); padding:0.25em; background:rgb(63,138,191); border-radius: 5px !important;text-decoration:none;"
+              <a href="<?= data[i].uns ?>">unsubscribe</a>
-                href="<?= data[i].uns ?>">unsubscribe</a>
-              <? } ?>
             </p>
+            <? } ?>
           </td>
-          <td>
+          <td class="email-subject">
-            <p style="margin: 0; padding: 0;">
+            <p style="margin: 0;">
-              <a style='font-size:1.5em !important; color:rgb(22,22,22) !important; text-decoration:none;' href="<?= data[i].permalink?>"><?= data[i].subject ?></a>
+              <a href="<?= data[i].permalink ?>"><?= data[i].subject ?></a>
             </p>
           </td>
         </tr>
       <? } ?>
-            
     </tbody>
   </table>
 </body>

noneroll.gs

@@ -1,109 +1,165 @@
-//run every hour to archive any new email from email in sheet
+// Global Constants
+const SHEET_ID = 'sheetID';                                       // Replace with your spreadsheet ID
+const SHEET_NAME = 'Sheet1';                                      // Update if your sheet is named differently
+const ZBULK_LABEL = 'zbulk';                                      // Change if you use a different Gmail label
+const DELAY_DAYS = 1;                                             // Number of days (1 = last 24 hours)
+
+// Helper to extract the email address from a sender string.
+function extractEmail(fromStr) {
+  const match = fromStr.match(/<([^>]+)>/);
+  return match ? match[1] : fromStr;
+}
+
+// Run every hour to archive any new email from email addresses in the sheet.
 function arch() {
-  var sheet = SpreadsheetApp.openById('sheetID');
+  const spreadsheet = SpreadsheetApp.openById(SHEET_ID);
-  var range = sheet.getDataRange();
+  const sheet = spreadsheet.getSheetByName(SHEET_NAME);
-  var values = range.getValues();
+  
-  values = [].concat.apply([], values);
+  // Retrieve emails from the first column and convert to a Set for efficient lookups.
-  //Logger.log(values)
+  const sheetData = sheet.getRange(1, 1, sheet.getLastRow(), 1).getValues().flat();
-  var threads = GmailApp.getInboxThreads();
+  const sheetEmails = new Set(sheetData);
-  var label = GmailApp.getUserLabelByName('zbulk')
+  
-  for (var i = 0; i < threads.length; i++) {
+  const threads = GmailApp.getInboxThreads();
-    if(threads[i].isInInbox()){
+  const label = GmailApp.getUserLabelByName(ZBULK_LABEL);
-      var msg = threads[i].getMessages()[0];
+  
-      var email = msg.getFrom().replace(/^.+<([^>]+)>$/, "$1");
+  for (const thread of threads) {
-      if(values.indexOf(email) > -1){
+    if (thread.isInInbox()) {
-        threads[i].addLabel(label);
+      const msg = thread.getMessages()[0];
-        threads[i].markRead();
+      const email = extractEmail(msg.getFrom());
-        threads[i].moveToArchive();
+      if (sheetEmails.has(email)) {
+        thread.addLabel(label);
+        thread.markRead();
+        thread.moveToArchive();
       }
     }
   }
 }
 
-//run every day to add any emails added to zbulk folder to sheet
+// Run every day to add any emails from the zbulk folder to the sheet.
 function addEmail() {
-  var sheet = SpreadsheetApp.openById('sheetID');
+  const spreadsheet = SpreadsheetApp.openById(SHEET_ID);
-  var range = sheet.getDataRange();
+  const sheet = spreadsheet.getSheetByName(SHEET_NAME);
-  var values = range.getValues();
+  
-  values = [].concat.apply([], values);
+  // Retrieve emails from the first column and convert to a Set for efficient lookups.
-  var label = GmailApp.getUserLabelByName('zbulk')
+  const sheetData = sheet.getRange(1, 1, sheet.getLastRow(), 1).getValues().flat();
-  var threads = label.getThreads();
+  const sheetEmails = new Set(sheetData);
-  for (var i = 0; i < threads.length; i++) {
+  
-    var msg = threads[i].getMessages()[0];
+  const label = GmailApp.getUserLabelByName(ZBULK_LABEL);
-    var email = msg.getFrom().replace(/^.+<([^>]+)>$/, "$1")
+  const threads = label.getThreads();
-    if(values.indexOf(email) < 0){
+  
-      sheet.appendRow([email])
+  // Collect new emails in an array for batch insertion.
+  const newEmails = [];
+  
+  for (const thread of threads) {
+    const msg = thread.getMessages()[0];
+    const email = extractEmail(msg.getFrom());
+    if (!sheetEmails.has(email)) {
+      newEmails.push([email]);
+      sheetEmails.add(email); // Update the set to avoid duplicates in this run.
     }
   }
+  
+  // Append all new emails at once.
+  if (newEmails.length > 0) {
+    const startRow = sheet.getLastRow() + 1;
+    const range = sheet.getRange(startRow, 1, newEmails.length, 1);
+    range.setValues(newEmails);
+  }
 }
 
+// Automatically deduplicate email addresses in your spreadsheet.
+function dedupeEmails() {
+  const spreadsheet = SpreadsheetApp.openById(SHEET_ID);
+  const sheet = spreadsheet.getSheetByName(SHEET_NAME);
+  
+  // Read all email addresses from column A.
+  const data = sheet.getRange(1, 1, sheet.getLastRow(), 1).getValues();
+  const uniqueEmails = [];
+  const seen = new Set();
+  
+  // Iterate over each row.
+  for (let i = 0; i < data.length; i++) {
+    let email = data[i][0].toString().trim();
+    if (email && !seen.has(email)) {
+      seen.add(email);
+      uniqueEmails.push([email]);
+    }
+  }
+  
+  // Clear the existing data and write back only unique emails.
+  sheet.clearContents();
+  if (uniqueEmails.length > 0) {
+    sheet.getRange(1, 1, uniqueEmails.length, 1).setValues(uniqueEmails);
+  }
+}
+
+// Retrieve emails from the last 24 hours that are still in the zbulk label.
 function getEmails() {
-  var delayDays = 2
+  const maxDate = new Date();
-  var maxDate = new Date();
+  maxDate.setDate(maxDate.getDate() - DELAY_DAYS);
-  maxDate.setDate(maxDate.getDate() - delayDays);
-  var label = GmailApp.getUserLabelByName("zbulk");
-  var threads = label.getThreads();
-  var data = []
-  for (var i = 0; i < threads.length; i++) {
-    if (threads[i].getLastMessageDate()>maxDate){
-      var d = {}
-      d.permalink = threads[i].getPermalink()
-      d.subject = threads[i].getFirstMessageSubject()
-      var from = threads[i].getMessages()[0].getFrom()
-      d.from = from.replace(/\"|<.*>/g,'')
-      d.date = threads[i].getLastMessageDate()
-      if (from.match(/<(.*)>/)!==null){
-        d.email = from.match(/<(.*)>/)[1]
-      } else {
-        d.email = from
-      }
-      d.uns = null
   
-      var uns = threads[i].getMessages()[0].getRawContent().match(/^list\-unsubscribe:(.|\r\n\s)+<(https?:\/\/[^>]+)>/im);
+  const label = GmailApp.getUserLabelByName(ZBULK_LABEL);
-      if(uns) {
+  const threads = label.getThreads();
-        d.uns = uns[uns.length-1]
+  const data = [];
+  
+  // Check every thread.
+  for (const thread of threads) {
+    const lastMsgDate = thread.getLastMessageDate();
+    if (lastMsgDate > maxDate) {
+      const msg = thread.getMessages()[0];
+      const permalink = thread.getPermalink();
+      const subject = thread.getFirstMessageSubject();
+      const from = msg.getFrom();
+      const fromClean = from.replace(/\"|<.*>/g, '');
+      const emailMatch = from.match(/<([^>]+)>/);
+      const email = emailMatch ? emailMatch[1] : from;
+      let unsubscribe = null;
+      
+      // Try to extract the unsubscribe link from the raw content.
+      const rawContent = msg.getRawContent();
+      const unsMatch = rawContent.match(/^list\-unsubscribe:(.|\r\n\s)+<(https?:\/\/[^>]+)>/im);
+      if (unsMatch) {
+        unsubscribe = unsMatch[unsMatch.length - 1];
       } else {
-        var rex = /.*?<a[^>]*href=["'](https?:\/\/[^"']+)["'][^>]*>(.*?[Uu]nsubscribe.*?)<\/a>.*?/gi
+        // If not found in raw content, search within the email body.
-        body_t = threads[i].getMessages()[0].getBody()
+        const body = msg.getBody();
-        while(u = rex.exec(body_t)){
+        const regex = /<a[^>]*href=["'](https?:\/\/[^"']+)["'][^>]*>(.*?)<\/a>/gi;
-          Logger.log("regmatch" + u.length)
+        let match;
-          if(u[0].toLowerCase().indexOf('unsubscribe')!==-1){
+        while ((match = regex.exec(body)) !== null) {
-            for(var j = u.length-1; j >=0; j--){
+          if (match[2].toLowerCase().includes('unsubscribe')) {
-              if(u[j].substring(0,4)=="http"){
+            unsubscribe = match[1];
-                d.uns=u[j]
+            break;
-                break
           }
         }
-            if(d.uns){
-              break
       }
       
-          }
+      data.push({
-        }
+        permalink,
-      }
+        subject,
-      data.push(d)
+        from: fromClean,
-    } else {
+        date: lastMsgDate,
-      break
+        email,
+        uns: unsubscribe
+      });
     }
   }
   return data;
 }
 
-
+// Run every day to send a summary email including emails from the last 24 hours.
-//run every day to send summary email including emails from last 24 hours
 function noneroll() {
-  emails = getEmails()
+  const emails = getEmails();
-  if(emails.length > 0) {
+  if (emails.length > 0) {
-    var date = Utilities.formatDate(new Date(), Session.getScriptTimeZone(), 'yyyy-MM-dd');
+    const date = Utilities.formatDate(new Date(), Session.getScriptTimeZone(), 'yyyy-MM-dd');
-    var subject = "Bulk Emails: " + date;
+    const subject = "Bulk Emails: " + date;
-    var t = HtmlService
+    const template = HtmlService.createTemplateFromFile('email');
-      .createTemplateFromFile('email');
+    template.data = emails;
-    t.data = emails;
+    const htmlBody = template.evaluate().getContent();
-    var hB = t.evaluate().getContent();
+    
     MailApp.sendEmail({
       to: Session.getActiveUser().getEmail(),
       subject: subject,
-      htmlBody: hB
+      htmlBody: htmlBody
     });
   }
 }