dlbass93 VIP
Total posts: 44
06 Aug 2013 21:42

I have been working on a project ( http://928registry.928oc.org ), and Cobalt 8 has been phenomenal. We are attempting to rapidly grow the project by importing a massive amount of data. I have been unable to figure out how the import process works with Cobalt 8, so I am reaching out for some assistance. If anyone is willing and able to help out, I would really appreciate talking to you. What we want to do may not even be feasible, below is a summary.

Import all Porsche 928s based on VIN number. We will gain access to a large amount of vehicle specific data from Porsche such as paint, option codes, transmission, and so on. We would like one record to be created per car, and for it to update any existing cars (if possible).

That is about it. If you are interested, please let me know.

Thanks guys,

Derrick

Last Modified: 02 Mar 2014


Sackgesicht VIP
Total posts: 1,636
09 Aug 2013 01:07

Derrick,

if your VIN list is an almost complete listing, then extract the option codes from there (4 letter) and see how many unique codes you will find. This should be the basis for your final option code list. You can then compare it against (your) existing list to get the description and missing codes.

With this approach you ensure later a smooth import, since you will not experience any missing option codes.


Sackgesicht VIP
Total posts: 1,636
09 Aug 2013 14:25

I am not sure about the checksum calculator, we are finding out if we can share it or not,

No need to share anything ..

If you want to replace the underscore with the checksum character just follow the computation here

A VIN decoder description is available in some of the links above ...

I am not sure about the checksum calculator, we are finding out if we can share it or not,

or get him to extract those option codes like you suggested

If you need help or instructions how to proceed, let me know...

I am not sure about the checksum calculator, we are finding out if we can share it or not,

or get him to extract those option codes like you suggested

how on earth would I go about adding a timestamp to it

There are a lot of ways to do it .. actually you have it already. When you create a record, Cobalt will store the date and also date when you modify it.

You might just display it like "Mileage : 34,000 as of modification time "

For the km /mls a possible solution would be a mandatory radio button with both options and nothing selected at the beginning. That would force the encoder to choose it. Now you know what the user has chosen and you can convert it to what ever you want with a customized template , or even display both values at the same time ...


dlbass93 VIP
Total posts: 44
11 Aug 2013 09:21

I got the option codes parsed into multiple columns, then organized them all back into a long single column. Over 400k codes. Then, removed duplicates, and left with only 78. That seems a bit strange, but it looks like our document only covers two types of codes, I and M codes. Now I have to figure out which ones are what and match up descriptions.

I have also been playing around with the separate Type for the option codes, I think I am making progress!

Thanks for tips on the mileage thing, I added the 'modification time' string. Once I learn to work with logic in the custom templates, I will be able to go crazy with this site. I have a list of logic I need to adjust in the templates, just haven't gotten that much knowledge yet.

Thanks for the help guys, we are getting there! Also, Cobalt, and the team behind it, are awesome!


Sackgesicht VIP
Total posts: 1,636
11 Aug 2013 11:51

and left with only 78

Sounds low, but if you did everything right, why not ....

When i played around with the excel demo you provided i separated the option codes first into multiple columns by fixed length (4 character) -- maybe 15 rows or so ... then started to remove duplicates per column ... moved the remaining codes into 1 column and removed duplicates again.

and left with only 78

I and M codes.

I believe those are some sort of internal identifiers. Based on what i saw, while looking for info, especially the plate image on of the sides, i would display (use) only the trailing 3 letters and forget about the leading I or M to avoid the confusion with the owners.

and left with only 78

I and M codes.

Once I learn to work with logic in the custom templates, I will be able to go crazy with this site.

The real power of Cobalt is in the templates. Cobalt will provide you an environment where all record data is ready for you to be used. You just have to understand when to use a field output template and when to customize an intro/full/form/markup or category template.

I'm sure you will always receive the needed support here on your "journey" ... :D


Sackgesicht VIP
Total posts: 1,636
11 Aug 2013 13:38

Thanks for tips on the mileage thing, I added the 'modification time' string.

I thought more on directly showing the date. :D

Something like :

Mileage : 56,000 km (June, 2013)

How to do it:

1) Read the documentation

Templates (general) and Article Templates

2) Find out the ID of your mileage field.

You can find it under TYPES -- FIELDS on the right side.

3) Make a copy of your record template

( to make sure, your modifications will not be overwritten with the next update)

4) Make necessary modification

You can access the record creation time through $item->created and the last modification through $item->modify.

Since it has the format of "2012-04-21 03:21:19", you might want to change it.

echo date('F, jS, Y',strtotime($item->modify) )

See date formats here

Look for

<?php echo $field->result; ?>

You might find it around line 171 and 210. It is there 2 times to handle normal display and fields organized by tabs.

change it to something like this:

<?php echo ($field->id == YOUR_ID ? $field->result.'('.date('F Y',strtotime($item->modify) ).')' : $field->result); ?>

Replace YOUR_ID with the id of your mileage field.

There would be more sophisticated (complex) and better solutions to this. This should just serve as an example.


dlbass93 VIP
Total posts: 44
11 Aug 2013 16:37

I re-did it, and this time the results were much better, I used your advice this time as well. My issue the first time was that the clipboard will only hold so much data, so I had to break it up into smaller chunks!

http://sdrv.ms/1bn74LR

We are working on getting the descriptions located, and I still have the european car list to extract.


Sackgesicht VIP
Total posts: 1,636
11 Aug 2013 18:14

http://sdrv.ms/1bn74LR

That looks quite good.

I forgot to mention to set the cell type to text (not default) when splitting, to avoid that leading "0" will be ignored. Now you have to edit your first 14 records and add the missing "0"


Sackgesicht VIP
Total posts: 1,636
11 Aug 2013 18:27

hmm .. i am just wondering... If this list http://sdrv.ms/1bn74LR was a result of the option code extraction from your 928 VIN list, how come it contains values like this?

They should not be in any 928 list. right?


Sackgesicht VIP
Total posts: 1,636
12 Aug 2013 20:40

I will also make the corrections you suggested above

I forgot to include the units in my example. If you have a field which holds the km/mls, then get the id of it and replace the UNITS_ID in the sample code below with your field id.

<?php echo ($field->id == YOUR_ID ? $field->result.' '.$item->fields_by_id[UNITS_ID]->result.' ('. date('F Y', strtotime($item->modify) ).')' : $field->result); ?>

dlbass93 VIP
Total posts: 44
07 Oct 2013 21:44

Hey guys, I have a short import run that I want to test out, but I am having bad luck. I have created my Import menu item, I have set the category to the right spot. When I go to Import, I can select my CSV, but the next button is disabled, and I can't make it past this point. Is there something obvious that I need to do to make it work. I am not sure if my import file is actually going to work, but I wanted to test it to see. Thanks,


dlbass93 VIP
Total posts: 44
08 Oct 2013 13:42

The attached zip contains two different CSV files that I tried to use. Both have the same result. I do not want to actually do a real import of either of these, just testing it. I still have a lot of work to do with the big file before it is ready. And the VIN only file wouldn't serve much purpose to import, just for testing. When viewed in notepad, the big file is definitely CSV, but maybe something about it is wrong. The VIN list is .csv, but there are no commas because it is a single column. Right?

Thanks guys,


Sackgesicht VIP
Total posts: 1,636
19 Oct 2013 17:16

Derrick,

i tried the 2MB file (928_US-Can_81-95.csv) of your attachment and it seems to work. After the successful parsing, it shows the fields and i can assign them ...

The small file will also work, but you should add a title row to it....


dlbass93 VIP
Total posts: 44
24 Oct 2013 17:10

Thank Sackgesicht, sorry for being so late to reply, I forgot to come and check...

Hmm, that got me to thinking about something I should have already checked. In Inspect Element in Chrome, under the Network tab, I get this output over and over once I upload my CSV. I checked the temp folder of my site, and it was 755, I made it 777 temporarily, and it is still doing this error. Joomla > System Information > Directory Permissions, reports that the temp folder is writable. Joomla 3.1.5 and Cobalt 8.345


dlbass93 VIP
Total posts: 44
12 Nov 2013 10:10

We think we have determined the error to be with permissions on the JSON files. We get this output when viewing inspect element.

[21:54:30.635] GET http://928registry.928oc.org/tmp/1384235542.json?dataType=json [HTTP/1.1 403 Forbidden 73ms]

I can see that the upload and extract is working properly, as there are files in those directories, but we do not get the success message for parsing the file. We tried the file on a different server and it worked properly.

I thought maybe my template was messing it up, but I switched to protostar, and it is still happening. I guess this means it is a server problem, but what?


Sergey
Total posts: 13,748
13 Nov 2013 00:21

What is your server? IIS or Apache?

Powered by Cobalt