After a day-ish of trying to figure out a nice clean ( Javascript-less ) way to make a queue to import images, I decided to just forget the queue and let Tanzawa import images as fast as possible. The number of simultaneous requests seems to be limited by the browser anyways.

The last thing I need to do is import individual post content. Maybe it's because I'm not building fun features, but this last Wordpress import feels like such a slog.

I made a fun hack for importing images. I'm using (part) of the Hotwire stack for the dynamic portions of Tanzawa. Most dynamic web applications today use client side rendering, which means the server sends a json data structure and your browser has code/templates/logic to instruct it how to turn it into html for display. Hotwire is "html over the wire", so all of your logic and rendering happens on the server and the browser just displays the result.

APIs traditionally return JSON. The image import api I wrote about yesterday also returned json, because that's what apis do. I was thinking the Javascript I need to write to update the page after an element has been imported when it hit me – if each photo in the list is wrapped in a turbo-frame, my api can return html, and all of my logic and rendering is in one place on the server. Perfect!

My plan was to then just write some Javascript that would traverse my list of images and call the api one by one. Then take the html response and replace the existing item.  But then I noticed that Turbo frames can have a source url i.e. I could put my import api url as the source for the frame and Turbo would automatically call it for me.

Turbo also support lazy loading. Which means that it's not going to load the frame until it shows up on the page. Which means I can import all of my images by just scrolling down the page.

So rather than have a bunch of Javascript to control a queue to make an api call to manually modify some html/css on the front end, I just have a list of images that you scroll down and each call is automatically made with the results are automatically updated on the page. Simple is best.

I broke the chain with a weekend off after about 3 months of working on Tanzawa a bit each and everyday. Today I'm back at it and I made a small api that imports images from Wordpress. Tomorrow I should be able to build a small interface that'll loop through the attachments and automatically download them.

Managed to get the category to stream and post kind  to (tanzawa) post kind mappings working. I also got the attachment import records saving properly. 

My basic plan for importing attachments is as follows. Each photo in Wordpress is exported as a post with the post type as "attachment". The guid for the item is the url for the attached file. So, I've created a record has a foreign key to the originating wordpress import record, the post guid, my own uuid, and a nullable foreign key to the resulting Tanzawa file attachment. 

Once the file has been imported, I'll have a Tanzawa file attachment set so I'll easily be able to pick up where I left off. 

Also since I'm keeping the originally uploaded Wordpress export file around and references between Tanzawa data and imported data, as I add features and capabilities to Tanzawa, I'll have the option to go back and pull in meta-data from Wordpress that I skipped on the initial import.

I've got the base interface for mapping categories to streams worked out. Adding a new stream directs you to the django admin, which isn't ideal from a user perspective, I kinda like it because modifying system data should be different.

Have got the base upload form working. After uploading the file it's automatically creating placeholder records for all post formats (Wordpress' build in Post Kind),  Categories, and Post Kinds. 

But thinking more about the actual worflow, I think it will be better not automatically create those records and truly split it separate steps. So after uploading the file you're taken to a list page with a list of all uploaded Wordpress files.

Next to each filename there will be five buttons: "Set Category Mapping", "Set Post Format Mapping" ,  "Set Post Kind Mapping" (if found), "Import Media", and "Import Posts".  The "Import Posts" button will be disabled until mapping has been setup and media has been imported.

Uploading the file will automatically redirect you to the "Set Category  Mapping" page, but if you leave the process midway through you'll be able to pickup where you left off.

Getting the base models used for importing Wordpress posts into Tanzawa built has made the task feel a bit less daunting. There's a clear path forward.

Parsing the Wordpress XML file with feedparser strips all of the Wordpress specific data. But it looks like I can use BeautifulSoup (which I'm using elsewhere) to get what I need. The "xml" parser preserves the CData, so I can get the encoded data, too. Progress.